2

enter image description here

In this picture you can see the measured datapoints (blue) and a curve I fit in (orange). The value with x=10000 seems like an outlier, and I am thinking about removing it, to get a better fitting curve. However, I don't want to remove it manually.

Methods I considered:

  • Trim at y<0.55. seems crude and unreliable, since the data can change.
  • remove points with a big vertical distance to the neighboring points. However, the vertical distance isn't that big, compared to the datapoints with bigger x.
  • IQR; only few datapoints, and the point isn't that far out.
  • fit curve iteratively and with each iteration remove one of the points. Subsequently observe how well the curve fits. With this Method I fear losing points which actually contribute valid information. Furthermore there could be more than one outlier.

Could anyone point me in the right direction? Any help is appreciated!

chl
  • 50,972
  • 18
  • 205
  • 364
Marco
  • 121
  • 2
  • 2
    What makes you think that point is an outlier rather than a point that doesn't conform to your model ? – deemel Nov 02 '20 at 09:17
  • Good Question! Yes, the data-point could be lower than it's neighbors. However to get to this dataset I have taken the mean of two points with same x. I know that sometimes I have some unreliable datapoints. I'm pretty confident that one of the two values was a bad one. As I am writing this I noticed I could filter the values before calculating a mean, and then maybe even omit the mean completely. – Marco Nov 02 '20 at 10:20
  • What exactly do you mean by "sometimes I have some unreliable datapoints " ? What makes you say that one of the two might be a "bad one" ? Are you confident that such data points stem from an error in the sampling? I'm aware this is somewhat off topic , I just get the impression that you're working with a biased few of your data. If you're interested , we can discuss this in a chat – deemel Nov 03 '20 at 07:51
  • I have never used the chat feature before, and haven't found a button to do so when visiting your profile. Do you mean https://chat.stackexchange.com/ ? – Marco Nov 03 '20 at 14:30
  • I created a chat room for this discussion, see https://chat.stackexchange.com/rooms/115827/outliers – deemel Nov 04 '20 at 06:58

0 Answers0