1

I conducted a two-way ANOVA, with perceived diagnosticity, i.e. the ability to evaluate haptic properties of a product, as the dependent variable and sound manipulation (congruent, incongruent and no-sound) and product stimulus (Handtuch and TV) as independet variables. The results are as follows:

Anova Table (Type III tests)

Response: PD
                       Sum Sq  Df  F value    Pr(>F)    
(Intercept)           2056.20   1 784.8304 < 2.2e-16 ***
Manipulation            40.70   2   7.7668 0.0006036 ***
Stimulus                 6.47   1   2.4692 0.1180734    
Manipulation:Stimulus    0.59   2   0.1126 0.8935742    
Residuals              419.19 160                       
---
Signif. codes:  
0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

enter image description here Since there is a significant effect of the sound manipulation, I would like to explore this further. I see three options: (1) planned comparisons, (2) simple effects and (3) post-hoc tests.

(1) planned comparisons

My hypothesis was, that a congruent sound would improve the perceived diagnosticity to a greater degree than no sound, or an incongruent sound. Thus it might make sense to compare the congruent sound, with the combination of no sound and incongruent sound, wouldn't it? Also would it make sense to then conduct any of the other tests?

(2) simple effects

Andy Field et al. (2012) seems to use simple effects, as a consequence of having had a significant interaction effect. This is not the case here. Yet would it not make sense to use simple effects and compare the congruent sound with the no-sound and incongruent sound separately? or any other comparison?

(3) post-hoc tests

Would it still make sense to conduct post-hoc tests, if I decide to go with any of the above options?

I am a bit lost, as to what tests make the most sense, from a reporting perseptive. Any thoughts are welcomed.

Jens Stach
  • 149
  • 2
  • 9

1 Answers1

0

Since you had a clear hypothesis, I would just consider the two post hoc t-tests: 1) congruent vs. no 2) congruent vs. incongruent, or even better you consider a model in which no/incongruent are on the same level (so it would be a 2x2 instead of a 2x3 model)

This because the more tests you make the more you increase the risk of false positives. To control this risk you will have to correct for the multiplicity of comparisons you are performing (eg through a Bonferroni correction) which ultimately will reduce the statistical power of your analysis.

fabiob
  • 622
  • 4
  • 13
  • thanks fabiob. As you suggested, planned comparisons probably make the most sense, i.e. combining no and incongruent into one group and comparing it with the congruent condition. Yet, one more thing. If I were also to conduct post-hoc tests, could I not cut the p-value in half, i.e. use a one-sided test, since I have a directional hypothesis? – Jens Stach May 07 '18 at 08:13
  • yes definitely. – fabiob May 07 '18 at 09:07