3

I used PCA to reduce the features of 1020 vector size into desired length like 32, 64, 128 etc.

Now I want to make LDA dimensionality reduction to compare them.

I'm stuck now can't figure out how do I reduce into desired feature vector length the original data set using LDA. (I want it to reduce it to 32, 64 etc).

I'm using the following MATLAB code to achieve it: LDA

The problem is that I have 5 classes and that code reduces the features size into 4. I tried to see what the code does but it seems the maximum what I can get is the feature size of 5.

Franck Dernoncourt
  • 42,093
  • 30
  • 155
  • 271
maximus
  • 224
  • 3
  • 7

1 Answers1

4

In LDA, unlike PCA, the maximum number of features after dimensionality reduction is $c - 1$, where $c$ is the number of classes. Since you have 5 classes, the maximum number of features is 4. You can learn the math here. On slide 23, the maximum rank of the between class scatter matrix is $S_B$ is $c-1$. So, the maximum number of eigenvalues/eigenvectors is $c-1$. This is why the maximum number of features is $c-1$.

Starz
  • 432
  • 1
  • 3
  • 11
  • Sorry, just found a more complete answer here: http://stats.stackexchange.com/questions/167690/linear-discriminant-analysis-newbie-question – Starz Feb 04 '17 at 16:01
  • Can I know that in the context of dimensionality reduction using LDA/FDA. `LDA/FDA can start with n dimensions and end with k dimensions, where k < n`. Is that correct? Or The output is `c-1 where c is the number of classes and the dimensionality of the data is n with n>c.` – aan May 06 '20 at 21:24