Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [convolution]

Convolution is a function-valued operation on two functions $f$ and $g$: $\int _{-\infty }^{\infty }f(\tau )g(t-\tau )d\tau$. Often used for obtaining the density of a sum of independent random variables. This tag should also be used for the inverse operation of deconvolution. DO NOT use this tag for convolutional neural networks.

Convolution is a function-valued operation on two functions $(f*g)(t) = \int _{-\infty }^{\infty }f(\tau )g(t-\tau )d\tau$. A common statistical application is obtaining the density of a sum of independent random variables. DO NOT use this tag for convolutional neural networks.

See https://en.wikipedia.org/wiki/Convolution and https://en.wikipedia.org/wiki/Deconvolution

429 questions
177
votes
8 answers

What does 1x1 convolution mean in a neural network?

I am currently doing the Udacity Deep Learning Tutorial. In Lesson 3, they talk about a 1x1 convolution. This 1x1 convolution is used in Google Inception Module. I'm having trouble understanding what is a 1x1 convolution. I have also seen this post…
jkschin
  • 1,873
  • 3
  • 9
  • 6
71
votes
4 answers

What is translation invariance in computer vision and convolutional neural network?

I don't have computer vision background, yet when I read some image processing and convolutional neural networks related articles and papers, I constantly face the term, translation invariance, or translation invariant. Or I read alot that the…
Hossein
  • 2,005
  • 3
  • 18
  • 32
54
votes
10 answers

Why is the sum of two random variables a convolution?

For long time I did not understand why the "sum" of two random variables is their convolution, whereas a mixture density function sum of $f(x)$ and $g(x)$ is $p\,f(x)+(1-p)g(x)$; the arithmetic sum and not their convolution. The exact phrase "the…
Carl
  • 11,532
  • 7
  • 45
  • 102
47
votes
6 answers

Importance of local response normalization in CNN

I've found that Imagenet and other large CNN makes use of local response normalization layers. However, I cannot find that much information about them. How important are they and when should they be used? From…
pir
  • 4,626
  • 10
  • 38
  • 73
32
votes
2 answers

Convolutional neural networks: Aren't the central neurons over-represented in the output?

[This question was also posed at stack overflow] The question in short I'm studying convolutional neural networks, and I believe that these networks do not treat every input neuron (pixel/parameter) equivalently. Imagine we have a deep network…
Koen
  • 421
  • 3
  • 4
31
votes
1 answer

"Kernel density estimation" is a convolution of what?

I am trying to get a better understanding of kernel density estimation. Using the definition from Wikipedia: https://en.wikipedia.org/wiki/Kernel_density_estimation#Definition $ \hat{f_h}(x) = \frac{1}{n}\sum_{i=1}^n K_h (x - x_i) \quad =…
Tal Galili
  • 19,935
  • 32
  • 133
  • 195
31
votes
7 answers

Convolutional Layers: To pad or not to pad?

AlexNet architecture uses zero-paddings as shown in the pic. However, there is no explanation in the paper why this padding is introduced. Standford CS 231n course teaches we use padding to preserve the spatial size: I am curious if that is the…
Jumabek Alihanov
  • 363
  • 1
  • 3
  • 9
19
votes
3 answers

A dynamical systems view of the Central Limit Theorem?

(Originally posted on MSE.) I have seen many heuristic discussions of the classical central limit theorem speak of the normal distribution (or any of the stable distributions) as an "attractor" in the space of probability densities. For example,…
symplectomorphic
  • 503
  • 2
  • 12
17
votes
4 answers

The sum of independent lognormal random variables appears lognormal?

I'm trying to understand why the sum of two (or more) lognormal random variables approaches a lognormal distribution as you increase the number of observations. I've looked online and not found any results concerning this. Clearly if $X$ and $Y$ are…
Patty
  • 1,599
  • 1
  • 8
  • 21
17
votes
2 answers

What does the convolution step in a Convolutional Neural Network do?

I am studying convolutional neural networks (CNNs) due to their applications in computer vision. I am already familiar with standard feed-foward neural networks, so I'm hoping that some people here can help me take the extra step in understanding…
ComputerScientist
  • 435
  • 1
  • 3
  • 11
16
votes
3 answers

How exactly do convolutional neural networks use convolution in place of matrix multiplication?

I was reading Yoshua Bengio's Book on deep learning and it says on page 224: Convolutional networks are simply neural networks that use convolution in place of general matrix multiplication in at least one of their layers. however, I was not 100%…
Charlie Parker
  • 5,836
  • 11
  • 57
  • 113
15
votes
2 answers

Sum of Bernoulli variables with different success probabilities

Let $x_i$ be independent Bernoulli random variables with success probabilities $p_i$. That is, $x_i=1$ with probability $p_i$ and $x_i=0$ with probability $1-p_i$. Is there a closed expression or an approximate formula for the distribution of the…
becko
  • 3,298
  • 1
  • 19
  • 36
15
votes
3 answers

Wouldn't multiple filters in a convolutional layer learn the same parameter during training?

Based from what I have learned, we use multiple filters in a Conv Layer of a CNN to learn different feature detectors. But since these filters are applied similarly (i.e. slided and multiplied to regions of the input), wouldn't they just learn the…
cjbayron
  • 343
  • 2
  • 9
15
votes
2 answers

Are there mathematical reasons for convolution in neural networks beyond expediency?

In convolutional neural networks (CNN) the matrix of weights at each step gets its rows and columns flipped to obtain the kernel matrix, before proceeding with the convolution. This is explained on a series of videos by Hugo Larochelle…
Antoni Parellada
  • 23,430
  • 15
  • 100
  • 197
14
votes
2 answers

Distribution of the convolution of squared normal and chi-squared variables?

the following problem came up recently while analyzing data. If the random variable X follows a normal distribution and Y follows a $\chi^2_n$ distribution (with n dof), how is $Z = X^2 + Y^2$ distributed? Up to now I came up with the pdf of…
Leo Szilard
  • 141
  • 4
1
2 3
…
28 29