Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [dirac-delta]

Dirac delta is a generalized function, or distribution, on the real number line that is zero everywhere except at zero, with an integral of one over the entire real line.

Dirac delta is a generalized function, or distribution, on the real number line that is zero everywhere except at zero, with an integral of one over the entire real line.

25 questions
10
votes
0 answers

What is the KL divergence of distribution from Dirac delta?

The Kullback–Leibler (KL) divergence of two continuous distributions $P(x)$ and $Q(x)$ is defined as $$D_{KL}(P \mid\mid Q) = \int_{X} P(x) \log{\left[\frac{P(x)}{Q(x)}\right]} dx$$ How can one compute the KL divergence when $Q$ is the Dirac delta…
saxen
  • 133
  • 7
10
votes
3 answers

Should Dirac's delta function be regarded as a subclass of the Gaussian distribution?

In Wikidata it is possible to link probability distributions (like everything else) in an ontology, e.g., that the t-distribution is a subclass of the noncentral t-distribution, see, e.g.,…
Finn Årup Nielsen
  • 284
  • 2
  • 7
8
votes
4 answers

Role of Dirac function in particle filters

Particle approximations to probability densities are often introduced as a weighted sum of Dirac functions $$p(x) \approx \sum_{i=1}^N \omega^i \delta(x-x^i)$$ with the weights $$\omega^i \propto \frac{p(x^i)}{q(x^i)}$$ normalized such that they…
Constantin
  • 1,117
  • 1
  • 9
  • 24
6
votes
2 answers

Dirac delta function in likelihood function

I have tried to understand this myself but what I have found on the internet so far has not helped. I have a likelihood function that for part of it has the following statement: d0 is the Dirac delta function at zero and it enters in the likelihood…
user3742335
  • 101
  • 2
5
votes
2 answers

Delta function in monte carlo sampling

I am confused by the dirac delta function in the formulation of monte carlo sampling. http://www.cs.ubc.ca/~arnaud/doucet_johansen_tutorialPF.pdf, for instance, defines in section 3.1 page 8 the marginal as…
Mike
  • 51
  • 2
4
votes
2 answers

Applying Bayes' rule in a more technical way when densities don't exist

Say $y \mid x \sim \text{Normal}(Ax, B)$ and $x \sim \text{Normal}(c,D)$. Let's assume further that $y \in \mathbb{R}^1$ and $x \in \mathbb{R}^2$. To find $p(x \mid y)$ we can usually do \begin{align*} p(x \mid y) &\propto p(y \mid x) p(x)…
Taylor
  • 18,278
  • 2
  • 31
  • 66
4
votes
1 answer

Dirac Delta function Notation

I am trying to understand the delta function notation used to be express a monte carlo approximation of a probability distribution. The notation used in this (p10) is $\pi(x_{1:n}) = \frac{1}{N}\sum^N_{i=1}\delta_{X^i_{1:n}}(x_{1:n})$ where…
user3235916
  • 517
  • 2
  • 12
4
votes
1 answer

Characteristic function of the Dirac delta?

What is the characteristic function of the Dirac delta function? Is it $e^{i*0}=1$?
Hansel
  • 43
  • 3
3
votes
1 answer

how to perform marginalization of posterior to find the predicative distribution for bayesian logistic regression

I want to know how to show that $$p(a) = \int \delta(a - w^T x) q(w) dw$$ is gaussian, where $q$ is gaussian and $x$ is fixed, and $\delta$ is the dirac function. Everything below is just some motivating background and thoughts on the problem. In…
shimao
  • 22,706
  • 2
  • 42
  • 81
3
votes
1 answer

Empirical probability and Dirac distribution

According to Deep Learning p.65(Ian Goodfellow and Yoshua Bengio and Aaron Courville, available online): (...) This can be accomplished by defining PDF using the Dirac delta function $\delta(x)$: $$p(x)=\delta(x-\mu)$$ (...) By defining $p(x)$ to…
mokebe
  • 243
  • 4
  • 11
1
vote
1 answer

How to implement a mixture model for Dirac Delta and Normal distributions?

How could I fit data with observations from one Dirac delta component and $n$ normal distributed components? Where $n$ usually is between 1 and 5. My prior knowledge is that one component really is a Dirac delta distribution. I know its location in…
Jona Engel
  • 111
  • 3
1
vote
0 answers

Metropolis-Hastings with a "Dirac transition"

I'm running the Metropolis-Hastings algorithm on a product space $\tilde E:=I\times E'$, where $I$ is a finite nonempty set and $E'=\bigcup_{i\in I}E'_i$ with $E'_i:=[0,1)^{d_i}$ for $i\in I$. Given the current state $\tilde x=(i,x')$, I'm…
0xbadf00d
  • 539
  • 3
  • 8
1
vote
0 answers

joint density of mixed RVs: A random vector and its cardinality

Suppose $X$ and $Y$ are two continuous random vectors and $ m = |Y|$ is a discrete random variable that denotes the size of the continuous random vector $Y$, i.e. the number of its columns (the cardinality) is random. $$p(\mathbf{Y}|\mathbf{X}) =…
sci9
  • 337
  • 3
  • 13
1
vote
1 answer

Expectation with delta function

In Bishop on page 219, the equality below is given. Unfortunately, I fail to see how one finds the mean $\mu_{a}$ of this distribution. I know the definition of expectation operator, but how does one obtain the most right equality in the bottom…
user173192
  • 41
  • 4
1
vote
0 answers

How to define general covariance with a dirac delta with correct units

I want to define the covariance of a variable, say $X$, which is the integral over $Y(t)$: $$ {\rm Cov}[X,X'] = \int\int {\rm Cov} [aY(t), a'Y(t')] dt dt'.$$ Just as an example, let the variable $Y$ be unitless, so that the total covariance has…
StevenMurray
  • 203
  • 1
  • 6
1
2