1

I'm trying to understand the Fishers LDA. In the book I am using there is explained that one has to maximize expression

$$\frac{\left( a^T\cdot \bar{x}_2-a^T\cdot \bar{x}_1 \right)^2}{a^T\cdot W\cdot a}$$

with respect to $a$, where: $a$ is the vector defining the direction at which data should be projected, $\bar{x}_1, \bar{x}_2$ are mean vectors of the classes, and $W$ is the covariance matrix between two classes.

I solved an example with two classes:

C1 = {{1, 2}, {2, 3}, {3, 3}, {4, 5}, {5, 5}};

C2 = {{1, 0}, {2, 1}, {3, 1}, {3, 2}, {5, 3}, {6, 5}};

and got the vector $a=\{-0.420776, 0.471853\}$. As far as I know this vector should be perpendicular to the discriminant line, but when I plot it, it is exactly the discriminant line. My questions are:

Is my solution fine? How to plot the obtained $a$ vector?

amoeba
  • 93,463
  • 28
  • 275
  • 317
Misery
  • 249
  • 2
  • 12
  • 1
    I can confirm your eigenvector `a`. If you now rescale it to sum of squares=1 you'll get `-.665556, .746347`. These are the cosines between the discriminant axis and the two variables' axes. Your `eigenvector * sqrt(11-2)` is the (unstandardized) discriminant coefficients ([read](http://stats.stackexchange.com/a/48859/3277)) that help to compute discriminant scores. And no, you are not correct saying `vector should be perpendicular to the discriminant line`: it defines the discriminant line. – ttnphns May 01 '15 at 15:45
  • 1
    [This answer](http://stats.stackexchange.com/a/22889/3277) and [this one](http://stats.stackexchange.com/a/13246/3277) display dicriminant axes drawn in space of the variables. The angles between the discriminant and the variable axes are the (normalized to SS=1) eigenvectors. Discriminant axes are shown tiled of points which are discriminant scores - the perpendicular coordinates of data points onto the discriminants. – ttnphns May 01 '15 at 15:55
  • @ttnphns Thank you for your comments. Could you tell me how to plot discriminant line properly, and project the points onto it? – Misery May 02 '15 at 10:24
  • 1
    If you want to plot as I did on the above linked answers. A discriminant score of a point is the coordinates of the point in the (centered) variables space multiplied by the scaled eigenvector `-.665556, .746347`. Right? But it is the hypotenuse, and its projection on a variable axis, the cathetus, is `hypothenuse*cos` where cos is again the value taken from `-.665556, .746347`. Easy. – ttnphns May 02 '15 at 11:59

0 Answers0