2

Participants in a study all respond to stimuli under two conditions, and some difference in response time is expected between Condition 1 and Condition 2.

I have three continuous outcome variables:
- A participant's average response time in Condition 1
- A participant's average response time in Condition 2
- The difference between C1 and C2, i.e. C1 minus C2

I also have 7 variables that are believed to predict these outcome variables. In particular, they're strongly believed to predict the difference score.

Is it appropriate to use multiple regression to predict a difference score of this sort, i.e. to do three regression analyses, one for each outcome variable?

What differences in interpretation will be required as a result of the fact that I'm predicting a difference score?

Are there any other things that I will need to handle differently in this context, for example in relation to regression assumptions and diagnostics?

user1205901 - Reinstate Monica
  • 11,303
  • 26
  • 77
  • 152

1 Answers1

3

You will get more precision and power by analyzing all the raw data, not just averages and not just differences in averages. This can be done using generalized least squares, mixed effects models, and Bayesian hierarchical models.

If you proceed with the current plan make sure that the response times are perfectly transformed so that differences have the desired meaning. Make a Tukey mean-difference plot (Bland-Altman plot) and check that the difference is unrelated to the average times and that variability is constant across levels of the average.

Frank Harrell
  • 74,029
  • 5
  • 148
  • 322
  • You say "check that the difference is unrelated to the average times and that variability is constant across levels of the average". Should that check be done subjectively by looking at the Tukey mean-difference plot (Bland-Altman plot)? Or is there some other test I should do in addition to looking at the plot? – user1205901 - Reinstate Monica Sep 08 '15 at 13:45
  • 2
    You can run a Spearman correlation but this is usually done subjectively. – Frank Harrell Sep 08 '15 at 14:07
  • I created a Bland-Altman plot [http://i.imgur.com/QvqXH48.png]. Subjectively it looks pretty good to me, but because I was playing with it before reading your response I ran a linear regression with the difference score C1-C2 as the outcome variable, and the mean score (C1+C2)/2 as the predictor. The predictor was marginally statistically significant, which I understand is undesirable. However, would you agree things look basically OK? – user1205901 - Reinstate Monica Sep 08 '15 at 14:16
  • 1
    The plot looks reasonable to me in terms of satisfying assumptions. Don't forget to consider analyzing the raw data though. – Frank Harrell Sep 08 '15 at 15:07
  • Thanks for the advice. I made a separate question for that topic, as I believe it is separate from the current question: http://stats.stackexchange.com/questions/171853/what-kind-of-linear-mixed-model-is-most-appropriate-for-this-data. – user1205901 - Reinstate Monica Sep 10 '15 at 08:05