I’m trying to train a CNN to categorize text by topic. When I use binary cross-entropy I get ~80% accuracy, with categorical cross-entropy I get ~50% accuracy. I don’t understand why this is. It’s a multiclass problem, doesn’t that mean that I have to use categorical cross-entropy and that the results with binary cross-entropy are … Read more
How do I initialize weights and biases of a network (via e.g. He or Xavier initialization)? 10 Answers 10
I have trained a binary classification model with CNN, and here is my code model = Sequential() model.add(Convolution2D(nb_filters, kernel_size[0], kernel_size[1], border_mode=”valid”, input_shape=input_shape)) model.add(Activation(‘relu’)) model.add(Convolution2D(nb_filters, kernel_size[0], kernel_size[1])) model.add(Activation(‘relu’)) model.add(MaxPooling2D(pool_size=pool_size)) # (16, 16, 32) model.add(Convolution2D(nb_filters*2, kernel_size[0], kernel_size[1])) model.add(Activation(‘relu’)) model.add(Convolution2D(nb_filters*2, kernel_size[0], kernel_size[1])) model.add(Activation(‘relu’)) model.add(MaxPooling2D(pool_size=pool_size)) # (8, 8, 64) = (2048) model.add(Flatten()) model.add(Dense(1024)) model.add(Activation(‘relu’)) model.add(Dropout(0.5)) model.add(Dense(2)) # define a … Read more
Why does zero_grad() need to be called during training? | zero_grad(self) | Sets gradients of all model parameters to zero. 5 Answers 5
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers. Want to improve this question? Update the question so it’s on-topic for Stack Overflow. Closed last year. Improve this question When I trained my neural network with Theano or Tensorflow, they will report a variable called “loss” per epoch. How … Read more
Closed. This question is opinion-based. It is not currently accepting answers. Want to improve this question? Update the question so it can be answered with facts and citations by editing this post. Closed 4 months ago. The community reviewed whether to reopen this question 4 months ago and left it closed: Original close reason(s) were … Read more
For any Keras layer (Layer class), can someone explain how to understand the difference between input_shape, units, dim, etc.? For example the doc says units specify the output shape of a layer. In the image of the neural net below hidden layer1 has 4 units. Does this directly translate to the units attribute of the … Read more
This question already has answers here: What are logits? What is the difference between softmax and softmax_cross_entropy_with_logits? (7 answers) Closed 1 year ago. In the following TensorFlow function, we must feed the activation of artificial neurons in the final layer. That I understand. But I don’t understand why it is called logits? Isn’t that a … Read more
I am trying to reconcile my understand of LSTMs and pointed out here in this post by Christopher Olah implemented in Keras. I am following the blog written by Jason Brownlee for the Keras tutorial. What I am mainly confused about is, The reshaping of the data series into [samples, time steps, features] and, The … Read more
What is the difference between ‘SAME’ and ‘VALID’ padding in tf.nn.max_pool of tensorflow? In my opinion, ‘VALID’ means there will be no zero padding outside the edges when we do max pool. According to A guide to convolution arithmetic for deep learning, it says that there will be no padding in pool operator, i.e. just … Read more
