5

I've been reading up a bit on eigenfaces. I think I understand the basic concept of it - vectorize a set of facial images then reduce the dimensionality of the images using PCA. What I don't really understand is the visualization of the lower-dimensional representation of the images.

In the facial images, the number of dimensions is the number of pixels so if you reduce the dimensionality of an image, you reduce the number of pixels. But then how do you visualize this image? Is it just a much smaller version of the full-dimensional original? The examples that I have seen do not look like this. Or do you alternatively make each pixel bigger so that the overall image is the same size as the original?

amoeba
  • 93,463
  • 28
  • 275
  • 317
user1893354
  • 1,435
  • 4
  • 15
  • 25
  • Take a look at some of the higher-voted [tag:pca] threads, such as http://stats.stackexchange.com/questions/2691. They might help you understand that the number of *coordinates* used to represent data may greatly exceed the number of *dimensions* needed to describe those data. Your question appears to conflate those two ideas. – whuber Oct 29 '13 at 19:38
  • I'm not sure what I am misunderstanding. Reducing the number of dimensions in an image reduces the number of pixels (doesn't it?). For example, applying the first x principal components to an image would make an image of lower dimension. mMy question is how to visualize this image – user1893354 Oct 29 '13 at 20:11
  • Thanks for the pointers! I have edited the answer and erased the false statement that faces are treated as features in the problem. – means-to-meaning Jan 19 '15 at 12:12

2 Answers2

5

Just a hint, after reading your comment. Each image (face) is represented as a stacked vector of length $N$. The different faces make up a dataset stored in a matrix $X$ of size $K\times N$. You might be confused about the fact that you use the PCA to obtain a set of eigenvectors (eigenfaces) $I = \{u_1, u_2, \ldots, u_D\}$ of the covariance matrix $X^TX$, where each $u_i \in \mathbb{R}^{N}$. You don't reduce the number of pixels used to represent a face, but rather you find a small number of eigenfaces that span a space which suitably represents your faces. The eigenfaces still live in the original space though (they have the same number of pixels as the original faces).

The idea is, that you use the obtained eigenfaces as a sort of archetypes that can be used to perform face detection.

Also, purely in terms of storage costs, imagine you have to keep an album of $K$ faces, each composed of $N$ pixels. Instead of keeping all the $K$ faces, you just keep $D$ eigenfaces, where $D \ll K$, together with the component scores and you can recreate any face (with a certain loss in precision).

amoeba
  • 93,463
  • 28
  • 275
  • 317
means-to-meaning
  • 1,581
  • 8
  • 13
  • Oh! I thought the pixels were the dimensions. This makes more sense. I guess I thought that reducing the dimensionality of an image meant reducing the number of pixels that it consisted of. – user1893354 Oct 29 '13 at 21:07
  • Wait, those are two different pairs of shoes. Of course does reducing the dimensionality of an image mean reducing the number of pixels. But the eigenfaces don't deal with reducing the dimensionality of an image, but with reducing the size of a set of images. – means-to-meaning Oct 29 '13 at 21:19
  • But couldn't you express a facial image in terms of, say, the first 3 eigenfaces? The result would be an image of reduced dimensionality, wouldn't it? – user1893354 Oct 29 '13 at 23:21
  • 2
    -1. This answer turns everything upside down and is VERY misleading! Check the answer by @Dima, it is correct. Pixels are dimensions. Eigenfaces are eigenvectors of the covariance matrix. Keeping $D$ eigenfaces means reducing the dimensionality from $N$ to $D$, but each of the $K$ faces is still there, described now by the $D$ numbers instead of by $N$ numbers. The number of faces $K$ is not reduced! I will be very happy to reverse my downvote if you edit to clarify. – amoeba Jan 18 '15 at 21:27
  • I will try to dissect it step by step to understand where the misleading part is. "...but each of the K faces is still there, described now by the D numbers instead of by N numbers" seems to say the same thing as "Instead of keeping all the K faces, you just keep D eigenfaces... and you can recreate any face". "The number of faces K is not reduced!", not sure what that means as by performing an SVD on your dataset, the original faces are not explicitly in the data anymore, they can be recreated though using the eigenfaces. I was pointing out the improvements in storage by using eigenfaces?! – means-to-meaning Jan 19 '15 at 09:34
  • @means, look [here on the wiki](http://en.wikipedia.org/wiki/Eigenface#Practical_implementation). It explicitly says that the covariance matrix is $N \times N$ size (in your notation) and that *"Each eigenvector has the same dimensionality (number of components) as the original images, and thus can itself be seen as an image"*. If the covariance matrix is of size $N$ (and you seem to agree?), then $N$ pixels are features, and not $K$ faces, as you claim. Covariance matrix must be the size of the number of features, be definition. – amoeba Jan 19 '15 at 10:56
  • 1
    You are right! I understand now that I muddled it all up and perhaps in looking for a confusing point, created a confusion myself. I think I got thrown off by the fact that most tutorials on PCA set up their matrix with images as column vectors and didn't realize that they compute the covariance matrix sucht that the pixels are the features. The dimensions in the original space are definitely the pixels. I rewrote the answer. The only thing is, now I don't see anything confusing about the eigenfaces the poster was originally referring to. Thanks for the comments! – means-to-meaning Jan 19 '15 at 12:09
  • Thanks! Now it's a good answer; I reverted my downvote and upvoted instead. I also edited for some minor improvement in your latex code, but more importantly, replaced "factor loadings" with "component scores". Please check that it makes sense. If eigenfaces are eigenvectors (proportional to loadings), then what you need to reconstruct the original faces are scores, not loadings. – amoeba Jan 19 '15 at 14:01
5

To clarify a bit more: in your original high-dimensional space, pixels are the dimensions. In the new space, each image is represented as a linear combination of a relatively small number of basis images, the eigenfaces. So in the new space, the eigenfaces are the dimensions.

Dima
  • 421
  • 2
  • 6