0

I have a repeated measures experiment where all participants completed several trials for each condition. My dependent variables are response time and accuracy. I am using the Interquartile Range as my outlier removal criteria. Before I ask my questions, here are a few details on the IQR:

The IQR represents the central 50 percent or the area between the 75th and 25th percentile of a distribution. Any point is an outlier if it is above the 75th percentile or below the 25th percentile by a factor of 1.5 times the IQR.

Here, we can describe our dataset with:

  1. Minimum (lowest) value of the data
  2. Quartile 1 = Q1 = first quartile = 25% of the data starting from the minimum value
  3. Quartile 2 = median = Q2 = midpoint of the dataset
  4. Quartile 3 = Q3 = 75% of the data starting from the minimum value
  5. Maximum value of the data set

The Interquartile Range = IQR = Q3 – Q1 = how the data are spread about the median. We then use the IQR to find the lower and upper bounds of our exclusion criteria:

  • lower: [Q1- (1.5)*IQR]
  • upper: [Q3+(1.5)*IQR]

Note: the 1.5 is a scale value. When you do the math, we can see that anything beyond 2.7 sigmas from the mean would be considered an outlier and this is closest to the Gaussian Distribution where an outlier is is any value beyond 3 sigmas of either side of the mean.

Here is a Wikipedia image (https://en.wikipedia.org/wiki/Interquartile_range):

enter image description here

My Questions:

Is it valid to remove trials as outliers using the IQR method? Is it more valid to (1) find the IQR of all subjects and then remove these trials or (2) to find the IQR of each subject and then remove each subject's respective outlier trials?

It seems more valid to remove trials rather than all of a single participants data because they may only have a few trials (of the many) that represent inattention (e.g., spaced out momentarily). And, these few trials might throw off their overall response time (or accuracy) measure.

Thanks for the input.

john connor
  • 21
  • 3
  • Please tell us the specifics of the "IQR method." – whuber Oct 10 '21 at 18:43
  • Thanks for the suggestion @whuber. I added a description of the IQR method to contextualize my question. – john connor Oct 10 '21 at 19:26
  • 2
    This amounts to removing outliers identified in a standard boxplot. Please, then, [search our site](https://stats.stackexchange.com/search?q=boxplot+outlier) for appropriate answers. The hit at https://stats.stackexchange.com/questions/259654 looks like a good place to start. – whuber Oct 10 '21 at 19:27
  • My questions is specific to whether it's valid to remove TRIALS using this method rather than PARTICIPANTS. And if so, should the IQR be found for each participant or for the whole sample size. It's not specific to whether IQR is a valid method itself or how to do the IQR method computations. – john connor Oct 10 '21 at 19:32
  • 3
    Why remove any points at all? The values you have are the values you measured, right? – Dave Oct 10 '21 at 19:37
  • Not all data is representative of what a person is doing. For example, 10 extreme RT data points that deviate from the remaining 150 data points (from one person) likely represents inattention (e.g., junk data). My question is not a debate about outlier removal philosophies. My question is about whether removing TRIALS from a participant is valid (in the literature) because removing all of a participant's data seems far too liberal of a criteria. – john connor Oct 10 '21 at 19:45
  • 4
    I think it is OK to remove for highly-suspected 'inattention'. But your removal based on IQR has an automated feel that worries me. Important to bear in mind that even moderately large computer-generated _normal_ samples often have a few outliers. Samples from moderately right-skewed distribution characteristically have high boxplot outliers. – BruceET Oct 11 '21 at 01:02
  • @BruceET: do you want to post your comment(s) as an answer? [Better to have a short answer than no answer at all.](https://stats.meta.stackexchange.com/a/5326/) Anyone who has a better answer can post it. – kjetil b halvorsen Oct 11 '21 at 14:02
  • 1
    It would definitely be wrong to remove trials based on a lumped analysis of all participants. There are many options available, but providing good advice comes down to understanding what you really hope to accomplish by removing some of your data. – whuber Oct 11 '21 at 14:11
  • Thanks @whuber. I want to remove response times from trials where they are obviously extreme and likely due to error (e.g., asking a question or momentarily not paying attention - both of which are common is psychometric studies). Using the IQR seems like one way to describe the reasoning of removal in a manuscript. The IQR seems like a decent enough method if you actually look at the underlying math and how it relates to a Gaussian distribution. Sounds like what you are suggesting is to find the IQR of each individual rather than the group, then remove based on their individual values. – john connor Oct 11 '21 at 14:35
  • Follow up -- assuming those values make sense to remove after looking at them in a visualization (said differently: not removing them for the sake of removing them, but checking that it makes sense to remove them). – john connor Oct 11 '21 at 15:19
  • 1
    I am not suggesting anything like that, because--as others have commented here--such automatic methods are often inferior or even misleading. Better potential alternatives include using robust statistical procedures that do not require preliminary outlier identification and removal. What is best, or even reasonable, to do depends on what analyses you intend to perform. – whuber Oct 11 '21 at 15:27
  • @kjetilbhalversen. I have a recent answer to a similar question [here](https://stats.stackexchange.com/questions/547801/tukeys-fences-for-outlier-removal/547805#547805). – BruceET Oct 11 '21 at 16:12
  • 1
    Thanks for your invitation to post my misgivings as an answer. But by now @whuber has made additional important points in comments, from which I would not want to detract attention. – BruceET Oct 11 '21 at 16:17
  • Outliers are real and an issue whether you leave them in or take them out. It's a complicated process and decision with a lot of debate around criteria - much of which could be addressed via transparency. Nonetheless, do you have suggestions on other robust statistical procedures? Are you referring to ANOVS or linear mixed models? I know that ANOVAs are very robust to assumption violations. Linear mixed models are good because they account for individual variability. @whuber – john connor Oct 11 '21 at 18:24

0 Answers0