Questions tagged [tensorflow]

A Python library for deep learning developed by Google. Use this tag for any on-topic question that (a) involves tensorflow either as a critical part of the question or expected answer, & (b) is not just about how to use tensorflow.

Tensorflow is an open source library that makes it easy to create ML models that can run in any environment. It has Python, C, C++, JavaScript and R APIs.

It offers multiple levels of abstraction which allows users to choose the right one for their use case. The high-level deep learning library Keras is fully integrated into Tensorflow, which makes getting started with TensorFlow and machine learning easy.

There are numerous extension libraries that helps build advanced models using and extending Tensorflow.

684 questions
77
votes
4 answers

Should I use a categorical cross-entropy or binary cross-entropy loss for binary predictions?

First of all, I realized if I need to perform binary predictions, I have to create at least two classes through performing a one-hot-encoding. Is this correct? However, is binary cross-entropy only for predictions with only one class? If I were to…
infomin101
  • 1,843
60
votes
1 answer

CNN architectures for regression?

I've been working on a regression problem where the input is an image, and the label is a continuous value between 80 and 350. The images are of some chemicals after a reaction takes place. The color that turns out indicates the concentration of…
rodrigo-silveira
  • 1,318
59
votes
6 answers

Adam optimizer with exponential decay

In most Tensorflow code I have seen Adam Optimizer is used with a constant Learning Rate of 1e-4 (i.e. 0.0001). The code usually looks the following: ...build the model... # Add the optimizer train_op =…
MarvMind
  • 693
  • 1
  • 6
  • 5
57
votes
4 answers

Is it possible to give variable sized images as input to a convolutional neural network?

Can we give images with variable size as input to a convolutional neural network for object detection? If possible, how can we do that? But if we try to crop the image, we will be loosing some portion of the image and if we try to resize, then, the…
Ashna Eldho
  • 671
57
votes
5 answers

How to weight KLD loss vs reconstruction loss in variational auto-encoder?

in nearly all code examples I've seen of a VAE, the loss functions are defined as follows (this is tensorflow code, but I've seen similar for theano, torch etc. It's also for a convnet, but that's also not too relevant, just affects the axes the…
memo
  • 1,019
57
votes
6 answers

Understanding LSTM units vs. cells

I have been studying LSTMs for a while. I understand at a high level how everything works. However, going to implement them using Tensorflow I've noticed that BasicLSTMCell requires a number of units (i.e. num_units ) parameter. From this very…
user124589
50
votes
1 answer

Step-by-step example of reverse-mode automatic differentiation

Not sure if this question belongs here, but it's closely related to gradient methods in optimization, which seems to be on-topic here. Anyway, feel free to migrate if you think some other community has better expertise in the topic. In short, I'm…
ffriend
  • 10,220
46
votes
1 answer

How is softmax_cross_entropy_with_logits different from softmax_cross_entropy_with_logits_v2?

Specifically, I suppose I wonder about this statement: Future major versions of TensorFlow will allow gradients to flow into the labels input on backprop by default. Which is shown when I use tf.nn.softmax_cross_entropy_with_logits . In the same…
Christian Eriksson
  • 573
  • 1
  • 4
  • 10
36
votes
3 answers

Building an autoencoder in Tensorflow to surpass PCA

Hinton and Salakhutdinov in Reducing the Dimensionality of Data with Neural Networks, Science 2006 proposed a non-linear PCA through the use of a deep autoencoder. I have tried to build and train a PCA autoencoder with Tensorflow several times but I…
Donbeo
  • 3,237
32
votes
3 answers

What's the difference between variance scaling initializer and xavier initializer?

In Tensorflow's implementation of ResNet , I find they use variance scaling initializer, I also find xavier initializer is popular. I don't have too much experience on this, which is better in practice?
Hanamichi
  • 683
32
votes
3 answers

Relu vs Sigmoid vs Softmax as hidden layer neurons

I was playing with a simple Neural Network with only one hidden layer, by Tensorflow, and then I tried different activations for the hidden layer: Relu Sigmoid Softmax (well, usually softmax is used in the last layer..) Relu gives the best train…
Ancalagon BerenLuthien
  • 2,002
31
votes
7 answers

Deep learning : How do I know which variables are important?

In terms of neural network lingo (y = Weight * x + bias) how would I know which variables are more important than others? I have a neural network with 10 inputs, 1 hidden layer with 20 nodes, and 1 output layer which has 1 node. I'm not sure how to…
user1367204
  • 1,011
31
votes
2 answers

Loss function for autoencoders

I am experimenting a bit autoencoders, and with tensorflow I created a model that tries to reconstruct the MNIST dataset. My network is very simple: X, e1, e2, d1, Y, where e1 and e2 are encoding layers, d2 and Y are decoding layers (and Y is the…
AkiRoss
  • 525
29
votes
3 answers

Difference between samples, time steps and features in neural network

I am going through the following blog on LSTM neural network: http://machinelearningmastery.com/understanding-stateful-lstm-recurrent-neural-networks-python-keras/ The author reshapes the input vector X as [samples, time steps, features] for…
Vipul Jain
  • 393
29
votes
2 answers

Is it common practice to minimize the mean loss over the batches instead of the sum?

Tensorflow has an example tutorial about classifying CIFAR-10 . On the tutorial the average cross entropy loss across the batch is minimized. def loss(logits, labels): """Add L2Loss to all the trainable variables. Add summary for for "Loss" and…
Clash
  • 773
1
2 3
…
45 46