0

I'm having some difficulty understanding the answer to my previous question, and I think part of the reason is I'm still fuzzy on one of the concepts/notation. I found another question where this occurs:

EM algorithm gaussian mixtures- derivation

What exactly is the meaning of $E_{\theta^{(t)}}[f(\theta)]$, where $\theta^{(t)}$ is a specific value of the parameter $\theta$? And does the result always depend only on $\theta$ but not on $\theta^{(t)}$?

Thanks

J.D.
  • 49
  • 8

1 Answers1

0

This usually is meant to denote an expectation which is conditioned on a certain starting condition. In your case, the probability used to compute the expectation depends on some $\theta$. So if you had to manually compute the expectation, you'd need to know the value of this parameter. The subscript then indicates that you should use the value of the parameter you have obtained at the step t of your algorithm (because in the EM algorithm you compute an optimizing value of the parameter for the expectation, but this expectation relies on having some previous estimate of the parameter to be computed and optimized).

Here I am using the term conditioning loosely. What I mean is say you are running an algo, at step t you have computed $\theta^(t)$.

Now, your situation is something like $$ \mathbb{E}_{\theta^{(t)}}[f(X,\theta)] = \sum_x \mathbb{P}_{\theta^{(t)}}(X = x) f(x,\theta) $$ Don't be confused by the param having the same name, the one raised to the power t is your previous estimate, the other is to found by optimizing wrt it.

Three Diag
  • 511
  • 2
  • 8
  • 1
    Usually, conditioning on a random variable is denoted with a midline, as in $E[f(\theta)\mid \theta=\theta^{(t)}].$ The subscripting generally refers to a *parameterized distribution* where $\theta^{(t)}$ denotes a value of the parameter. That's not the same thing as a conditional expectation, because the parameter need not be (and often is not) a random variable. – whuber Apr 22 '20 at 13:18
  • So you're saying it's the expectation when $\theta$ is taken to be $\theta^{(t)}$? If so, then shouldn't the result be an expression containing only $\theta^{(t)}$ and not $\theta$? But that's not what happened in the answer to my question here: https://stats.stackexchange.com/questions/461367/exponential-reestimation-formula-in-em-algorithm/461468 There, the answerer wrote $Q(t,t')=E_t[\text{something involving }t']$, and on the next line gave an expression with both $t$ and $t'$ in it. – J.D. Apr 22 '20 at 14:42
  • Here I am using the term conditioning loosely. What I mean is say you are running an algo, at step t you have computed $\theta^t=3$. Now, the expectation $E_{\theta^t} f(X) = sum_x P_{\theta^t} (x) f(x)$ is computed using your estimate at time t (as that is the param of the probability). Now, your situation is something like $E_{\theta^t} f(x, \theta) = \sum_x P_{\theta^t} (x) f(x, \theta)$. Don't be confused by the param having the same name, the one raised to the power t is your previous estimate, the other is to found by optimizing wrt it. – Three Diag Apr 23 '20 at 14:10