2

Suppose that I have two IVs (AnxietyLevel & experimental conditions (dummy coded as 0 and 1), and two control variables (Age & Gender) and one DV (mean score on a questionnaire). And I am trying to find out both main and interaction effects via hierarchical regression analysis.

My models for the above in R would be: 

H1 <- lm(mean_score ~ age + gender, data = df)

where gender shows a significant p-value.

However, the effect of gender becomes insignificant with the model below:

H2 <- lm(mean_score ~ age + gender + anxiety * expconditions, data = df)

So now nothing, even the interaction effect, is significant. What does this mean in plain English (if possible)?

I still cannot seem to wrap my head around it, so any help will be appreciated!!

user240313
  • 89
  • 1
  • 7
  • There are multiple things that can happen to create tis. A reproducible example would help explain it. – Jeremy Miles May 22 '19 at 17:34
  • I tried to give a little more specific example, hope this helps! – user240313 May 22 '19 at 17:40
  • How large is your sample? Maybe when you add the interaction term there is no enought df to have reliable p-values... – maiava May 22 '19 at 17:45
  • Ah yes my sample size is quite low for an interaction effect (n = 95) perhaps that is why.. – user240313 May 22 '19 at 17:47
  • Thanks! But then I am curious, if there was a large number of sample, and the the main effect became insignificant with an insignificant interaction effect, what would it mean..? – user240313 May 22 '19 at 17:47
  • You have to ensure at least 10-15 observations for each predictor (each level of the categorical count as on predictor). – maiava May 22 '19 at 17:51
  • 1
    When you add the interaction effect and the curves of each level of the factor have opposite directions, the main effect tends to be insignificant, maybe is your case, but the interaction is not significant... it can happen – maiava May 22 '19 at 17:53
  • Is anxiety a mediator here ? What is the causal hypothesis ? – Robert Long May 22 '19 at 18:10
  • One of the hypothesis is that how the anxiety affects the mean score would depend on the experimental condition that the participant was assigned to. – user240313 May 22 '19 at 18:11

1 Answers1

0

You should also try an AICc approach. The AICc will tell you the best combination of predictors, maybe the AICc indicates that you should remove some variable to improve your model.

Try this:

library(MuMIn)
model <- lm(mean_score ~ age + gender + anxiety * expconditions, data = df) #the full model or global model

dredge(model)

You can stay with the models with deltaAICc <2 or 4, see HERE

You can also average the selected models through the model.avg function from MuMIn package.

Karolis Koncevičius
  • 4,282
  • 7
  • 30
  • 47
maiava
  • 109
  • 5
  • Mhm, I was indeed trying to compare the models, but I encountered a problem in R (which I posted about on the StackOverflow page but still without luck)... And It looks that the package you recommended is not avilable for the R I have (for R version 3.3.3). I guess I first need to fnd out why r keeps giving me an error. Thanks! – user240313 May 22 '19 at 17:54
  • update your R for the updated version... send me you question regarding model selection please – maiava May 22 '19 at 18:15
  • Just updated the R to the latest one, but still says that it is not available, strange... – user240313 May 22 '19 at 18:37