9

My experiment had a 2x3 design with one covariate. If I analyse the results using an ANOVA, I get a strongly significant interaction between the two main factors (p<.001). If I add the covariate into the analysis and do an ANCOVA, the interaction between the two main factors is not significant (p=.252).

The covariate itself was not significant, nor did it have any significant interactions.

There are similar questions on this website, but I was hoping to understand what does such a result pattern say about the covariate? How should I interpret the results?

Adding the covariate was the novel feature of my study and therefore it is important to me to understand this interaction properly.

Is the covariate influencing the results?

EDIT:

I've been thinking about this all day and I guess the possible interpretation would be: When controlling for the covariate, the interaction of the main effects is no longer significant.

In which case I don't understand a couple of things:

a) Does that mean the covariate is responsible for the interaction?

b) Why is the interaction between FactorA*FactorB*Covariate not significant then?

c) Why isn't the covariate significantly interacting with anything at all, when its presence seems to be influencing the results?

EDIT 2:

Maybe I am doing something wrong, because it is not making much sense to me. Here are my results:

http://imageshack.us/g/580/ancova1.png/

The last two pictures show when I run an ANOVA without the covariate (repeated measures, Factor 1 (2 levels). Factor 2 (3 levels).

The first three when I run an ANCOVA on the same factors, except I put in OptimismScore as a covariate.

Tomas Engelthaler
  • 91
  • 3
  • What happens if you run the same model, with the covariate as the outcome? – Jeremy Miles Apr 29 '13 at 17:18
  • The covariate itself is not significant (p=.622) – Tomas Engelthaler Apr 29 '13 at 18:44
  • Maybe paste your code to make sure we've understood it. The effect of the covariate is constrained to be the same for all outcomes (if I've understood correctly) and therefore this is a bit strange. – Jeremy Miles Apr 29 '13 at 19:01
  • Updated the post with a link to my results. – Tomas Engelthaler Apr 29 '13 at 19:24
  • I guess my only question is: From the results, what can we say about the covariate? What implications can we draw from the data set when it comes to the role of the covariate? – Tomas Engelthaler Apr 29 '13 at 19:37
  • 3
    Think of this: suppose the covariate were practically *equal* to the interaction. Then, upon introducing the covariate, there would be no need for the interaction at all. – whuber Apr 29 '13 at 19:40
  • I see. I haven't thought about that before. Is there a statistical way of determining how similar the covariate is to the interaction? – Tomas Engelthaler Apr 29 '13 at 19:51
  • See also this question which at least in the title is asking the same question: http://stats.stackexchange.com/questions/57423/why-is-anova-interaction-no-longer-significant-after-including-a-covariate/57433#57433 – Jeromy Anglim Apr 30 '13 at 06:30
  • 3
    Interaction terms are sometimes correlated with omitted main effects. I think it's best to get the main effects right before looking at interactions. – Frank Harrell Jul 29 '13 at 12:18
  • Centering the covariate fixed the problem. – Tomas Engelthaler Apr 30 '13 at 05:19
  • If you have time and inclination, I believe adding a little explanation would help future readers of this thread. Also, you can accept your reply to close the thread. – chl Apr 30 '13 at 08:05
  • 2
    If you did it correctly, centering should have no effect on interaction term estimates and their standard errors. Centering should only affect main effects. – Frank Harrell Jul 29 '13 at 14:57

1 Answers1

0

Ordinarily I would say that each term you add diminishes the residual degrees of freedom, and that in turn means your F-statistic has to be larger in order to attain a given level of significance. That does not appear to be the case here because your F-values themselves are very different when you include the covariate.

Some statistical pakcages get higher order F-value calculations wrong in the presence of covariate interactions (specifically, what goes into the denominator). One way to sanity test this is to obtain the coefficient estimates for the model with and without the covariate and see if any of the estimates change a lot (several-fold) between the two. That would be an indicator that the covariate really does make a difference.

But I also am mystified why centering the covariate would fix this. That's weird enough that if I were you I wouldn't rely on these calculations until speaking to a statistician.

f1r3br4nd
  • 2,084
  • 18
  • 31