18

I know regular PCA does not follow probabilistic model for observed data. So what is the basic difference between PCA and PPCA? In PPCA latent variable model contains for example observed variables $y$, latent (unobserved variables $x$) and a matrix $W$ that does not has to be orthonormal as in regular PCA. One more difference that I can think of regular PCA only provide principal components, where PPCA provides the probabilistic distribution of the data.

Could someone please through more light on the differences between PCA and PPCA?

amoeba
  • 93,463
  • 28
  • 275
  • 317
Vendetta
  • 545
  • 2
  • 4
  • 12
  • 2
    See [here](http://www.di.ens.fr/~fbach/courses/fall2005/Bishop_Tipping_1999_Probabilistic_PCA.pdf). – Ami Tavory Oct 17 '16 at 07:34
  • 1
    +1. See my answers here http://stats.stackexchange.com/questions/208731 and also here http://stats.stackexchange.com/questions/203087. Have you read the slides that you linked to? They seem to explain everything in detail. Can you follow that exposition or is it too complicated? – amoeba Oct 17 '16 at 10:00
  • @amoeba, I followed the slides I get some differences however It does not give me clear intuition that what PPCA can do that PCA can't do ? What happens technically by introducing latent variables? Estimation of covariance as in PPCA can be done in regular PCA too? If you can add an answer it would be really helpful – Vendetta Oct 18 '16 at 00:43
  • @amoeba, Those two questions are pretty well answered. Particularly the question on principal subspace in probabilistic PCA. This gives me more intuition in understanding the estimation of principal components from W. – Vendetta Oct 18 '16 at 01:04
  • OK, I will try to post an answer, but I am quite busy these days. I will try to find time this week, but might postpone it till next week. (+1 by the way) – amoeba Oct 18 '16 at 22:14
  • Hmm, looks like I forgot about it last year :-) Looks like your question remains quite popular so maybe I should post an answer after all... – amoeba Oct 09 '17 at 21:31

1 Answers1

10

The goal of PPCA is not to give better results than PCA, but to permit a broad range of future extensions and analysis. The paper states some of the advantages clearly in the introduction, ie/eg:

"the definition of a likelihood measure enables a comparison with other probabilistic techniques, while facilitating statistical testing and permitting the application of Bayesian models".

Bayesian models in particular are enjoying a huge renaissance lately, eg VAE, "auto-encoding variational Bayes", https://arxiv.org/abs/1312.6114 . Extension of PCA to be usable in variational frameworks and similar has the potential for another researcher to say 'Oh hey, what if I do ... ?'

Hugh Perkins
  • 4,279
  • 1
  • 23
  • 38