Lets say I have a factor called cognitive load that has 3 levels (low, medium, high) and I want to determine how load levels can affect reaction time on some task
There are at least 2 ways that I could conduct this test.
The first would be a one way ANOVA with a single 3 level factor. After running this test I find that the main effect of this factor is not significant. I was always taught that without a significant main effect you shouldn't/dont need to test the differences between each of the groups/levels
The second way is to dummy code the factor and treat it as a regression, predicting reaction time from the 2 generated dummy variables. If I treat low load as my reference group, then my dummy variables would capture the difference between low/medium and low/high. When I conduct this analysis I find that one of my coefficients is significant (e.g. high is significantly different from low load)
So this situation leads to 2 different interpretations of the results. On the one hand I dont have a significant effect of the factor overall, but upon looking at group differences with regression I find something interesting
Which technique should actually be followed? Should I be looking at group differences in ANOVA despite the lack of a significant main effect?
