0

I am experimenting PCA to detect outliers based on the reconstruction error.

What I do: I start with a 6 dimensions dataset and reduce it to 5 dimensions. Then, I reconstruct the initial dataset and end up with a reconstruction error for each observation. The observations associated with the highest reconstruction error will be defined as outliers.

The variables in my initial dataset have very low linear correlation (the maximum for two of them is 0.4).

Question 1: does it make sense to detect outliers with PCA if I have very low linear correlations between my variables?

I tried to draw an example in 2D to illustrate my concerns. The two variables I simulated have very low correlation (=0.25). The PCA would consist in reducing the 2 dimensions to 1 dimension only by projecting the points into the direction given by the eigenvector with the highest magnitude (eigenvalue). Say the green eigenvector has the highest magnitude. Then the highest projection errors will be associated with the points in the orange circles.

The problem here is that I am missing a large number of outliers that are situated in other locations. Moreover, the method seems not well adapted with this data shape

Question 2: any suggestion/idea on how I could identify whether I am in such case?

savoga
  • 1
  • 1
  • I'm having trouble seeing what connection correlation and "outliers" might have. As an example, consider a 2D dataset of 94 values evenly spaced along the line segment from $(-1,-1)$ to $(1,1)$ together with the two points $\pm(4,-4).$ The latter two are obvious outliers, but the correlation coefficient of this dataset is essentially zero. Part of the problem seems to be that you are expressing contradictory senses of "outlier:" what exactly is your definition? What are these "large number ... in other locations"?? – whuber Nov 22 '21 at 15:38
  • Maybe I shouldn’t even mention outliers here. Essentially, my question is: does this make sense to use PCA with such a non linear distribution? But to answer your question more precisely, an outlier is for me an observation that is far away from the others. In other words, it’s a data point that does not follow the same distribution as the others. I do not make any connection between outliers and correlation. – savoga Nov 22 '21 at 17:38
  • And regarding my sentence "I am missing a large number of outliers that are situated in other locations" I meant points that are very isolated from others such as (-375, -180) or (40, -110). – savoga Nov 22 '21 at 17:44
  • Please explain the sense in which such points (especially the second) might be considered outliers. Neither looks like one in your plot--they certainly are not far from their neighbors or many other points, at least in the implied Euclidean distance. – whuber Nov 22 '21 at 18:20
  • These points can be seen as outliers as they don't have close points in their neighbourhood. – savoga Nov 22 '21 at 19:23
  • That would be an unusual definition of an "outlier." Some people have termed such points "inliers." Unless you can offer a better characterization of what you mean, though, your question might be difficult to answer. – whuber Nov 22 '21 at 20:39
  • Aren't "reconstruction errors" the PC scores of the left-out 6th PC? As seen e.g. on Fig 2 [here](https://stats.stackexchange.com/a/288646/3277). – ttnphns Nov 22 '21 at 23:06
  • Thanks for the comments. I feel like the conversation is going away from the question I had in mind; I will remove the post and try to create a new one with a better formulation. – savoga Nov 23 '21 at 09:56

0 Answers0