Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [deep-learning]

An area of machine learning concerned with learning hierarchical representations of the data, mainly done with deep neural networks.

1888 questions
177
votes
8 answers

What does 1x1 convolution mean in a neural network?

I am currently doing the Udacity Deep Learning Tutorial. In Lesson 3, they talk about a 1x1 convolution. This 1x1 convolution is used in Google Inception Module. I'm having trouble understanding what is a 1x1 convolution. I have also seen this post…
jkschin
  • 1,873
  • 3
  • 9
  • 6
161
votes
11 answers

What is the difference between a neural network and a deep neural network, and why do the deep ones work better?

I haven't seen the question stated precisely in these terms, and this is why I make a new question. What I am interested in knowing is not the definition of a neural network, but understanding the actual difference with a deep neural network. For…
Nicolas
  • 1,781
  • 3
  • 10
  • 14
137
votes
5 answers

Why normalize images by subtracting dataset's image mean, instead of the current image mean in deep learning?

There are some variations on how to normalize the images but most seem to use these two methods: Subtract the mean per channel calculated over all images (e.g. VGG_ILSVRC_16_layers) Subtract by pixel/channel calculated over all images (e.g. CNN_S,…
Max Gordon
  • 5,616
  • 8
  • 30
  • 51
136
votes
4 answers

What is the difference between convolutional neural networks, restricted Boltzmann machines, and auto-encoders?

Recently I have been reading about deep learning and I am confused about the terms (or say technologies). What is the difference between Convolutional neural networks (CNN), Restricted Boltzmann machines (RBM) and Auto-encoders?
RockTheStar
  • 11,277
  • 31
  • 63
  • 89
120
votes
6 answers

Why are neural networks becoming deeper, but not wider?

In recent years, convolutional neural networks (or perhaps deep neural networks in general) have become deeper and deeper, with state-of-the-art networks going from 7 layers (AlexNet) to 1000 layers (Residual Nets) in the space of 4 years. The…
Karnivaurus
  • 5,909
  • 10
  • 36
  • 52
101
votes
6 answers

How is it possible that validation loss is increasing while validation accuracy is increasing as well

I am training a simple neural network on the CIFAR10 dataset. After some time, validation loss started to increase, whereas validation accuracy is also increasing. The test loss and test accuracy continue to improve. How is this possible? It seems…
Konstantin Solomatov
  • 1,203
  • 2
  • 10
  • 8
70
votes
2 answers

What is the difference between a neural network and a deep belief network?

I am getting the impression that when people are referring to a 'deep belief' network that this is basically a neural network but very large. Is this correct or does a deep belief network also imply that the algorithm itself is different (ie, no…
Vincent Warmerdam
  • 1,129
  • 1
  • 9
  • 10
65
votes
7 answers

Why is the validation accuracy fluctuating?

I have a four layer CNN to predict response to cancer using MRI data. I use ReLU activations to introduce nonlinearities. The train accuracy and loss monotonically increase and decrease respectively. But, my test accuracy starts to fluctuate wildly.…
Raghuram
  • 763
  • 1
  • 6
  • 10
59
votes
6 answers

Difference between "kernel" and "filter" in CNN

What is the difference between the terms "kernel" and "filter" in the context of convolutional neural networks?
ryguy
  • 791
  • 1
  • 6
  • 6
59
votes
4 answers

Recurrent vs Recursive Neural Networks: Which is better for NLP?

There are Recurrent Neural Networks and Recursive Neural Networks. Both are usually denoted by the same acronym: RNN. According to Wikipedia, Recurrent NN are in fact Recursive NN, but I don't really understand the explanation. Moreover, I don't…
crscardellino
  • 855
  • 2
  • 8
  • 10
58
votes
6 answers

Adam optimizer with exponential decay

In most Tensorflow code I have seen Adam Optimizer is used with a constant Learning Rate of 1e-4 (i.e. 0.0001). The code usually looks the following: ...build the model... # Add the optimizer train_op =…
MarvMind
  • 683
  • 1
  • 6
  • 5
58
votes
3 answers

Why do Convolutional Neural Networks not use a Support Vector Machine to classify?

In recent years, Convolutional Neural Networks (CNNs) have become the state-of-the-art for object recognition in computer vision. Typically, a CNN consists of several convolutional layers, followed by two fully-connected layers. An intuition behind…
Karnivaurus
  • 5,909
  • 10
  • 36
  • 52
56
votes
8 answers

R libraries for deep learning

I was wondering if there's any good R libraries out there for deep learning neural networks? I know there's the nnet, neuralnet, and RSNNS, but none of these seem to implement deep learning methods. I'm especially interested in unsupervised…
Zach
  • 22,308
  • 18
  • 114
  • 158
56
votes
4 answers

What is the definition of a "feature map" (aka "activation map") in a convolutional neural network?

 Intro Background Within a convolutional neural network, we usually have a general structure / flow that looks like this: input image (i.e. a 2D vector x) (1st Convolutional layer (Conv1) starts here...) convolve a set of filters (w1) along the…
Atlas7
  • 663
  • 1
  • 6
  • 7
55
votes
5 answers

Using deep learning for time series prediction

I'm new in area of deep learning and for me first step was to read interesting articles from deeplearning.net site. In papers about deep learning, Hinton and others mostly talk about applying it to image problems. Can someone try to answer me can it…
Vedran
  • 651
  • 1
  • 6
  • 4
1
2 3
…
99 100