0

I want to calculate the KL-Divergence between a Laplacian density g and a normal density f. I can decompose $KL(G|F)$ to $\mathbb{E}_g[\log g(X)]-\mathbb{E}_g[\log f(X)]$. I am already stuck with my math at $\mathbb{E}_g[\log g(X)]$.

We have $g(x)=\frac{1}{2\tau}\cdot\exp(-\frac{|x-\nu|}{\tau})$ and $f(x)=\frac{1}{2\pi\sigma^2}\cdot\exp(-\frac{(x-\mu)^2}{2\sigma^2})$.

$\log g(X)=-\log(2\tau)-\frac{|x-\nu|}{\tau}$, thus

$\mathbb{E}_g[\log g(X)]=\int_{-\infty}^{\infty}(-\log(2\tau)-\frac{|x-\nu|}{\tau})\cdot(\frac{1}{2\tau}\cdot\exp(-\frac{|x-\nu|}{\tau}))dx=$

$-\log(2\tau)\int_{-\infty}^{\infty}\frac{1}{2\tau}\exp(-\frac{|x-\nu|}{\tau})dx-\int_{-\infty}^{\infty}\frac{|x-\nu|}{2\tau^2}\exp(-\frac{|x-\nu|}{\tau})dx=$

$-\log(2\tau)-\int_{-\infty}^{\infty}\frac{|x-\nu|}{2\tau^2}\exp(-\frac{|x-\nu|}{\tau})dx$

How can I calculate that integral? And is it even right until there?

Joe_base
  • 105
  • 8
  • Looks like you may have things backward, and with a sign flip. If you want to write it in terms of expectations, $D_{KL}(f \parallel g) = E_{f(x)} \big[ \log f(x) \big] - E_{f(x)} \big[ \log g(x) \big]$ – user20160 Jun 27 '20 at 19:31
  • Thanks, I corrected the sign and the subscripts: $E_g[\log g(X)]$ and later $E_g[\log f(X)]$ are the quantities I want to know. – Joe_base Jun 27 '20 at 20:17
  • 1
    You may assume $\nu=0$ and $\tau=1$ by applying the general results at https://stats.stackexchange.com/questions/415435. At that point you can write down the value of the integral with no more work. – whuber Jun 27 '20 at 20:26
  • What you've written now is $D_{KL}(g \parallel f)$, not $D_{KL}(f \parallel g)$ – user20160 Jun 27 '20 at 21:43

0 Answers0