Questions tagged [convolution]

Convolution is a function-valued operation on two functions $f$ and $g$: $\int _{-\infty }^{\infty }f(\tau )g(t-\tau )d\tau$. Often used for obtaining the density of a sum of independent random variables. This tag should also be used for the inverse operation of deconvolution.

DO NOT use this tag for convolutional neural networks.

Convolution is a function-valued operation on two functions $$(f*g)(t) = \int _{-\infty }^{\infty }f(\tau ) \, g(t-\tau ) \, \mathrm d\tau.$$ A common statistical application is obtaining the density of a sum of independent random variables. DO NOT use this tag for convolutional neural networks.

See Convolution and Deconvolution.

481 questions
217
votes
8 answers

What does 1x1 convolution mean in a neural network?

I am currently doing the Udacity Deep Learning Tutorial. In Lesson 3, they talk about a 1x1 convolution. This 1x1 convolution is used in Google Inception Module. I'm having trouble understanding what is a 1x1 convolution. I have also seen this post…
jkschin
  • 2,293
  • 3
  • 13
  • 7
89
votes
6 answers

What is translation invariance in computer vision and convolutional neural network?

I don't have computer vision background, yet when I read some image processing and convolutional neural networks related articles and papers, I constantly face the term, translation invariance , or translation invariant . Or I read alot that the…
Hossein
  • 2,455
73
votes
10 answers

Why is the sum of two random variables a convolution?

For long time I did not understand why the "sum" of two random variables is their convolution , whereas a mixture density function sum of $f(x)$ and $g(x)$ is $p\,f(x)+(1-p)g(x)$ ; the arithmetic sum and not their convolution. The exact phrase "the…
Carl
  • 13,730
50
votes
7 answers

Importance of local response normalization in CNN

I've found that Imagenet and other large CNN makes use of local response normalization layers. However, I cannot find that much information about them. How important are they and when should they be used? From…
pir
  • 5,176
37
votes
2 answers

"Kernel density estimation" is a convolution of what?

I am trying to get a better understanding of kernel density estimation. Using the definition from Wikipedia: https://en.wikipedia.org/wiki/Kernel_density_estimation#Definition $$ \hat{f_h}(x) = \frac{1}{n}\sum_{i=1}^n K_h (x - x_i) \quad =…
Tal Galili
  • 22,224
36
votes
7 answers

Convolutional Layers: To pad or not to pad?

AlexNet architecture uses zero-paddings as shown in the pic. However, there is no explanation in the paper why this padding is introduced. Standford CS 231n course teaches we use padding to preserve the spatial size: I am curious if that is the only…
Jumabek Alihanov
  • 413
34
votes
2 answers

Convolutional neural networks: Aren't the central neurons over-represented in the output?

[This question was also posed at stack overflow] The question in short I'm studying convolutional neural networks, and I believe that these networks do not treat every input neuron (pixel/parameter) equivalently. Imagine we have a deep network (many…
Koen
  • 441
26
votes
4 answers

The sum of independent lognormal random variables appears lognormal?

I'm trying to understand why the sum of two (or more) lognormal random variables approaches a lognormal distribution as you increase the number of observations. I've looked online and not found any results concerning this. Clearly if $X$ and $Y$ are…
Patty
  • 1,799
25
votes
4 answers

Probability that sum of binary variables is even

Let $S_i \in \{0,1\}$ , $i=1,\dots,N$ be $N$ independent random binary variables, each taking the value 1 with probability $0 \le p_i \le 1$ (and the value 0 with probability $1-p_i$ ). I am interested in the sum, $$X = \sum_{i=1}^N S_i$$ Because…
a06e
  • 4,686
25
votes
3 answers

A dynamical systems view of the Central Limit Theorem?

(Originally posted on MSE.) I have seen many heuristic discussions of the classical central limit theorem speak of the normal distribution (or any of the stable distributions) as an "attractor" in the space of probability densities. For example,…
symplectomorphic
  • 593
  • 3
  • 12
21
votes
2 answers

Sum of Bernoulli variables with different success probabilities

Let $x_i$ be independent Bernoulli random variables with success probabilities $p_i$. That is, $x_i=1$ with probability $p_i$ and $x_i=0$ with probability $1-p_i$. Is there a closed expression or an approximate formula for the distribution of the…
a06e
  • 4,686
21
votes
2 answers

What does the convolution step in a Convolutional Neural Network do?

I am studying convolutional neural networks (CNNs) due to their applications in computer vision. I am already familiar with standard feed-foward neural networks, so I'm hoping that some people here can help me take the extra step in understanding…
ComputerScientist
  • 485
17
votes
1 answer

Distribution of sum of exponentials

Let $X_1$ and $X_2$ be independent and identically distributed exponential random variables with rate $\lambda$ . Let $S_2 = X_1 + X_2$ . Q: Show that $S_2$ has PDF $f_{S_2}(x) = \lambda^2 x \text{e}^{-\lambda x},\, x\ge 0$ . Note that if events…
SecretAgentMan
  • 1,655
17
votes
3 answers

Wouldn't multiple filters in a convolutional layer learn the same parameter during training?

Based from what I have learned, we use multiple filters in a Conv Layer of a CNN to learn different feature detectors. But since these filters are applied similarly (i.e. slided and multiplied to regions of the input), wouldn't they just learn the…
cjbayron
  • 383
17
votes
2 answers

Are there mathematical reasons for convolution in neural networks beyond expediency?

In convolutional neural networks (CNN) the matrix of weights at each step gets its rows and columns flipped to obtain the kernel matrix, before proceeding with the convolution. This is explained on a series of videos by Hugo Larochelle here :…
Antoni Parellada
  • 27,650
1
2 3
…
32 33