2

I am trying to analyze data from a factorial design experiment with three factors: Nitrogen, Phosphorus and CO2, with 2 levels each. I ran a three-way ANOVA in R:

> fit = aov( Chl ~ (Nitrogen*Phosphorus*CO2), data=rr2)
> print(drop1(fit,~.,test="F"))
Single term deletions

Model:
Chl ~ (Nitrogen * Phosphorus * CO2)
                        Df Sum of Sq    RSS     AIC F value  Pr(>F)  
<none>                               21.168 -203.21                  
Nitrogen                 1   0.88545 22.053 -200.13  4.8523 0.02959 *
Phosphorus               1   0.07483 21.242 -204.77  0.4101 0.52321  
CO2                      1   0.01695 21.185 -205.11  0.0929 0.76110  
Nitrogen:Phosphorus      1   0.85426 22.022 -200.30  4.6814 0.03254 *
Nitrogen:CO2             1   0.02174 21.189 -205.08  0.1192 0.73057  
Phosphorus:CO2           1   0.00376 21.171 -205.19  0.0206 0.88608  
Nitrogen:Phosphorus:CO2  1   0.20799 21.376 -204.00  1.1398 0.28791  
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Since CO2 appears to have no effect, does it make sense to simplify the model like so, discarding the CO2 variable:

> fit = aov( Chl ~ (Nitrogen*Phosphorus), data=rr2)
> print(drop1(fit,~.,test="F"))
Single term deletions

Model:
Chl ~ (Nitrogen * Phosphorus)
                    Df Sum of Sq    RSS     AIC F value    Pr(>F)    
<none>                           23.289 -199.37                      
Nitrogen             1   2.11369 25.402 -190.59 10.8912 0.0012725 ** 
Phosphorus           1   0.20896 23.498 -200.26  1.0767 0.3015159    
Nitrogen:Phosphorus  1   3.11497 26.404 -185.80 16.0506 0.0001074 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
ekatko1
  • 110
  • 9

1 Answers1

2

In my view the answer is NO and YES — depending on your aims:

No: If the aim of your study was to test the effect of all factors on Chl in a Null-Hypothesis-Significance-Testing (NHST) sense, i.e. if the aim was to confirm/reject any main or interaction effect of Nitrogen/Phosphorus/CO2 on Chl. In this case altering your analysis model post-hoc will likely lead to spurious significances due to "the garden of forking paths problem": If you allow post-hoc changes in analysis, you inflate Type I error, as each of the numerous possible analysis choices states a multiple-comparison.

I can think of two cases where simplifying the model is ok:

Yes 1: If the aim of your second analysis is merely to complement your main analysis and to show that the results hold true even when simplifying the model then that is ok (when clearly labeled as a post-hoc analysis).

Yes 2: If the purpose of your experiment was to explore potential relationships between Chl and Nitrogen/Phosphorus/CO2 to find the optimal model ("model building"), then systematically dropping or adding variables may make sense. However, you should conduct an independent study to test the resulting model before arriving at any final conclusions.

There is a similar post discussing this question for linear regression. All points raised there apply here alike since both linear regression/ANOVA/ANCOVA are basically variations of general linear model analysis.

Community
  • 1
mzunhammer
  • 1,118
  • 7
  • 18