2

I have a time series and wish to estimate its entropy, is it correct to use ksdensity function in Matlab to get estimates of the pdf at 100 points and then plug these into the standard Shannon entropy formula?

seanysull
  • 123
  • 5
  • 2
    Since a [density is not a probability](http://stats.stackexchange.com/questions/4220/can-a-probability-distribution-value-exceeding-1-be-ok/4223?s=6|0.0000#4223), the answer surely is no. – whuber Dec 09 '16 at 17:12
  • Any advice on how to estimate entropy of a time series or where to find the right info? – seanysull Dec 09 '16 at 18:25

1 Answers1

0

Depends on which Shannon entropy formula you use. Broadly, $H = \mathrm{E}[\mathrm{I}(X)]$. In the discrete case, this would be $H = \mathrm{E}[-\ln P(X)]$ or $$-\sum \limits_{i=1}^{n} P(x_i)\ln P(x_i)$$ This would make sense if you're using a histogram estimate of $P(X)$.

The kernel density estimate generates an estimate of the (continuous) density function $f(X)$. A such, you'd need a continuous estimate of $H = \mathrm{E}[\mathrm{I}(X)] = \mathrm{E}[-\ln f(X)]$ or $$-\int f(x)\ln f(x) dx$$ Since the kernel density estimate outputs discrete points $x_i, f(x_i)$, it would make sense to use something like trapezoid or Simpson's rule to get the integral.

I recommend consulting other sources (such as this one) to determine the best approximation method.

ssokolen
  • 116