I have read a couple of references on the interpretation of a latent variable in the GMM/EM literature and I found a massive discrepancy between the authors so much so I now have no idea how GMM/EM works.
Here is the first set of definition:
The latent variable is a discrete random variable that takes on values $z \in \{1,\ldots, K\}$ and is said to have categorical distribution, where $P[z = k] = \pi_k$
See page 26: https://www.cs.toronto.edu/~urtasun/courses/CSC411_Fall16/13_mog.pdf
https://ermongroup.github.io/cs228-notes/learning/latent/
http://www2.stat.duke.edu/~sayan/Sta613/2018/lec/emnotes.pdf
https://arxiv.org/pdf/1712.01263.pdf
And this question: Latent variable in Gaussian Mixture Model
Here is the second set of definition:
The latent variable is a discrete random variable $z$ that is a one-hot encoded vector, e.g., [1,0,0], [0,1,0], [0,0,1], where each component of $z = (z_k)$ is an indicator variable $z_k \in \{0, 1\}$ and we write $P[z_k = 1] = \pi_k$
See: M. P. Deisenroth, A. A. Faisal, C. S. Ong, "Mathematics of Machine Learning" page 363-364
Bishop: page 39 of these notes https://www2.cs.sfu.ca/~oschulte/teaching/726/spring11/slides/mychapter9.pdf
http://pillowlab.princeton.edu/teaching/statneuro2018/slides/notes16_LatentsAndEM.pdf
https://arxiv.org/pdf/2001.02923.pdf
And this question Question about the latent variable in EM algorithm
So the problem to me is that sometimes people are writing $P[z = k] = \pi_k$, and other times people are writing $P[z_k = 1] = \pi_k$.
What is going on?
