Why is ANOVA in R showing a different result than ANOVA from another source?

Question

I am new to R and statistics both. I am trying to perform 2x2 ANOVA on an excel dataset where the 2 factors are Gender and Athletes and the dependent variable is Anger Expression. When I am calculating ANOVA on the dataset the Gender Sum Sq always shows 1 for some reason, however when I check on http://vassarstats.net it gives different summary which is actually the correct one. Could you please help me with this?

I have tried to change the Gender and Athletes values from M/F & S/NS to 1/2. Also tried to switch the columns in the hope that something might change. I checked on vassarstats.net by entering the same values which gives me the correct answer. Dataset - https://drive.google.com/file/d/15cR4OR1hdso9Xm6fKc5jiFsSDzJK5pOn/view?usp=sharing

Code that I have executed

> Angry_moods_r$Gender <- as.factor(Angry_moods_r$Gender)
> Angry_moods_r$Athletes <- as.factor(Angry_moods_r$Athletes)
> anova1 <- aov(Anger_Expr ~ Gender+Athletes+Gender*Athletes, data = Angry_moods_r)
> summary(anova1)
                Df Sum Sq Mean Sq F value  Pr(>F)   
Gender           1      1     1.4   0.009 0.92598   
Athletes         1   1357  1357.2   8.709 0.00424 **
Gender:Athletes  1      5     5.2   0.034 0.85505   
Residuals       74  11532   155.8                   

Correct answer

                Df   Sum Sq Mean Sq F value  Pr(>F)   
Gender           1    20.87   20.87   0.13  0.7195   
Athletes         1  1286.93 1286.93   8.26  0.0053 **
Gender:Athletes  1     5.24    5.24   0.03  0.863   
Residuals       74 11532.19  155.84 
Total           77    12896

Answer 1

In R you can obtain the result like this:

# the data transformed to +/- values 
#      this will give a different behaviour for dropping a
#      fixed effect term while keeping the interaction term
a <- (0.5-(data$Athletes == 'S'))
g <- (0.5-(data$Gender == 'M'))
y <- data$Anger_Expr 

# linear model
m <- lm(y~a*g)

# the sum of squares by using type III sums
> drop1(m,.~.)
Single term deletions

Model:
y ~ a * g
       Df Sum of Sq   RSS    AIC
<none>              11532 397.70
a       1   1286.93 12819 403.95
g       1     20.87 11553 395.84
a:g     1      5.24 11537 395.74

The reason for these differences is that the analysis of variance can be performed in different ways.

1. You are comparing models with and without the factor, but this is ambiguous (see the different type I/II/III sums).

2. In addition the interaction term can be defined in various ways and this influences the difference in the sum of squared residuals when dropping a fixed effect term. This explains the difference between the answer from BruceET and the comment from Roland.

Answer 2

This is an unbalanced design. That is, you do not have the same number of replicates in each of the four cells. In Minitab, the 'Balanced ANOVA' procedure (correctly) shows an error because of the imbalance.

Below is output from the general linear model procedure in Minitab 17, which handles unbalanced designs.

Analysis of Variance

Source             DF   Adj SS   Adj MS  F-Value  P-Value
  Athletes          1   1286.9  1286.93     8.26    0.005
  Gender            1     20.9    20.87     0.13    0.715
  Athletes*Gender   1      5.2     5.24     0.03    0.855
Error              74  11532.2   155.84
Total              77  12896.0

My guess is that the R procedure you are using is not appropriate for unbalanced designs. Perhaps see this link about two-factor ANOVA in R --at the end, where unbalanced designs are discussed.

---

Notes: (1) If you just look at Mood scores and Athlete types (*53 'NS' and 25 'S'), you can do a two-sample t test. Results from a Welch (separate variances) test are as follows:

T-Test of difference = 0 (vs ≠): 
T-Value = 3.21  P-Value = 0.002  DF = 57

The corresponding test for Gender (48 'F' and 30 'M') does not show a significant difference.

(2) You can see the imbalance because there are unequal sample sizes in the two t tests. If this were a balanced design, the rows of the ANOVA table (other than Total) would correspond to orthogonal subspaces of a 77-dimensional vector space and (assuming normality) their SS's would be independent statistics. The Adjustments in SS's reflect compromises based on non-orthogonal projections.

(3) I did not check for normality or equal variances because the main issue of your Question is to explain differences in output between two software programs, But you should check ANOVA assumptions before interpreting results.