1

I have some measurements (concentration) made in 4 groups (W, X, Y, Z) and time is my covariate. I make a linear model:

fit <- lm(concentration~group*year, data=data)

The results are as follows: ANOVA table:

anova(fit)

Analysis of Variance Table

Response: concentration
           Df Sum Sq Mean Sq F value    Pr(>F)    
group       3 3600.7 1200.22 32.6132 4.081e-10 *** #!
year        1  559.7  559.71 15.2087 0.0004311 ***
group:year  3   97.3   32.42  0.8809 0.4607155    
Residuals  34 1251.3   36.80                      
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

and pairwise comparison:

summary(fit)
Call:
lm(formula = concentration ~ group * year, data = data)

Residuals:
   Min     1Q Median     3Q    Max 
-8.818 -4.019 -0.276  4.181 13.097 

Coefficients:
              Estimate Std. Error t value Pr(>|t|)
(Intercept)  -433.0108   828.4293  -0.523    0.605
groupX      -1574.0090  1170.3741  -1.345    0.188 #!
groupY      -1666.3673  1170.3741  -1.424    0.164 #!
groupZ      -1201.2766  1170.3891  -1.026    0.312 #!
year            0.2418     0.4128   0.586    0.562
groupX:year     0.7937     0.5831   1.361    0.182
groupY:year     0.8409     0.5831   1.442    0.158
groupZ:year     0.6104     0.5831   1.047    0.303

Residual standard error: 6.066 on 34 degrees of freedom
Multiple R-squared:  0.7729,    Adjusted R-squared:  0.7261 
F-statistic: 16.53 on 7 and 34 DF,  p-value: 2.852e-09

Now I have a problem in the interpretation of this data. As far as I understand, since the interaction in the ANOVA table is nonsignificant, I can check the group effect, and it is significant. This means that the intercept in different groups should be [significantly] different. But when I look to the summary table, there is no significant difference, at least – between group W and others (groupX, groupY and groupZ are nonsignificant). If I change the compared group from W to X or Y or Z the comparison results are still nonsignificant:

data2 <- data
data2$group[data2$group=="X"] <-"A"
fit <- lm(concentration~group*year, data=data2)
summary(fit)

Call:
lm(formula = concentration ~ group * year, data = data2)

Residuals:
   Min     1Q Median     3Q    Max 
-8.818 -4.019 -0.276  4.181 13.097 

Coefficients:
              Estimate Std. Error t value Pr(>|t|)  
(Intercept) -2.007e+03  8.267e+02  -2.428   0.0206 *
groupW       1.574e+03  1.170e+03   1.345   0.1876 #! 
groupY      -9.236e+01  1.169e+03  -0.079   0.9375 #! 
groupZ       3.727e+02  1.169e+03   0.319   0.7518 #!
year         1.035e+00  4.119e-01   2.514   0.0168 *
groupW:year -7.937e-01  5.831e-01  -1.361   0.1824  
groupY:year  4.717e-02  5.825e-01   0.081   0.9359  
groupZ:year -1.834e-01  5.825e-01  -0.315   0.7549  
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 6.066 on 34 degrees of freedom
Multiple R-squared:  0.7729,    Adjusted R-squared:  0.7261 
F-statistic: 16.53 on 7 and 34 DF,  p-value: 2.852e-09

data2 <- data
data2$group[data2$group=="Y"] <-"A"
fit <- lm(concentration~group*year, data=data2)
summary(fit)

Call:
lm(formula = concentration ~ group * year, data = data2)

Residuals:
   Min     1Q Median     3Q    Max 
-8.818 -4.019 -0.276  4.181 13.097 

Coefficients:
              Estimate Std. Error t value Pr(>|t|)  
(Intercept) -2.099e+03  8.267e+02  -2.539   0.0158 *
groupW       1.666e+03  1.170e+03   1.424   0.1636 #! 
groupX       9.236e+01  1.169e+03   0.079   0.9375 #! 
groupZ       4.651e+02  1.169e+03   0.398   0.6933 #! 
year         1.083e+00  4.119e-01   2.628   0.0128 *
groupW:year -8.409e-01  5.831e-01  -1.442   0.1584  
groupX:year -4.717e-02  5.825e-01  -0.081   0.9359  
groupZ:year -2.305e-01  5.825e-01  -0.396   0.6948  
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 6.066 on 34 degrees of freedom
Multiple R-squared:  0.7729,    Adjusted R-squared:  0.7261 
F-statistic: 16.53 on 7 and 34 DF,  p-value: 2.852e-09

data2 <- data
data2$group[data2$group=="Z"] <-"A"
fit <- lm(concentration~group*year, data=data2)
summary(fit)

Call:
lm(formula = concentration ~ group * year, data = data2)

Residuals:
   Min     1Q Median     3Q    Max 
-8.818 -4.019 -0.276  4.181 13.097 

Coefficients:
              Estimate Std. Error t value Pr(>|t|)
(Intercept)  -433.0108   828.4293  -0.523    0.605
groupX      -1574.0090  1170.3741  -1.345    0.188 #!
groupY      -1666.3673  1170.3741  -1.424    0.164 #!
groupZ      -1201.2766  1170.3891  -1.026    0.312 #!
year            0.2418     0.4128   0.586    0.562
groupX:year     0.7937     0.5831   1.361    0.182
groupY:year     0.8409     0.5831   1.442    0.158
groupZ:year     0.6104     0.5831   1.047    0.303

Residual standard error: 6.066 on 34 degrees of freedom
Multiple R-squared:  0.7729,    Adjusted R-squared:  0.7261 
F-statistic: 16.53 on 7 and 34 DF,  p-value: 2.852e-09

How is it possible that there are no significant difference between any two groups when there is a significant group effect? Apparently my interpretation that significant group effect means that at least one group differ significantly from other in the intercept value is incorrect. So what is the correct interpretation of the significant group effect?

Marta Cz-C
  • 225
  • 1
  • 7

1 Answers1

3

You have been caught in the trap of the difference between anova() and lm() in the ways that they partition variance among the predictor variables.

The anova() function uses Type I sums of squares, as explained in this answer. In this approach, you look at the predictor variables in the order that they are entered into the model. In your case, you first determine how much variance is explained by the set of group variables, then how much of what remains is explained by time, then how much of the residual is explained by the interaction terms.

In the summary of an lm() model with interactions, each level of the categorical variables, all continuous variables, and all interaction terms are examined in parallel, evaluating each of them with all the others taken into account. This can, as you see, lead to apparently different results from anova() even with the same underlying model. Further complicating the interpretation is, as you show, the default display with comparisons against the reference level of a categorical variable. Thus even the results for your continuous predictor, year, differ depending on which value of the categorical group variable is chosen as reference; as you are including interaction terms, the displayed result for year in summary(lm()) will be the coefficient for year for the reference category, not overall.

For the case at hand, as the anova() shows no significant interaction term you should be OK with continuing your analysis on a linear model without interactions, and proceeding with standard post-hoc tests for your groups. That said, you should check whether the time-course nature of your data might be violating the assumptions of independent observations that are required for standard interpretation of p-values. Following values over time can lead to data correlation structures that require formal analysis as repeated measures or time series.

Also, note that there seems to be an error in your very last displayed analysis. You seem to have tried to set "Z" to the reference by changing its label to "A", but it looks like "W" remained as the reference in the summary output. The relevel() function in R allows you to change the reference level without changing the labels themselves.

Community
  • 1
EdM
  • 57,766
  • 7
  • 66
  • 187