As far as I understand, $R^2$ explains how well the model predicts the observation. Adjusted $R^2$ is the one that takes into account more observations (or degrees of freedom). So, Adjusted $R^2$ predicts the model better? Then why is this less than $R^2$? It appears it should often be more.
Asked
Active
Viewed 6.4k times
3 Answers
31
$R^2$ shows the linear relationship between the independent variables and the dependent variable. It is defined as $1-\frac{SSE}{SSTO}$ which is the sum of squared errors divided by the total sum of squares. $SSTO = SSE + SSR$ which are the total error and total sum of the regression squares. As independent variables are added $SSR$ will continue to rise (and since $SSTO$ is fixed) $SSE$ will go down and $R^2$ will continually rise irrespective of how valuable the variables you added are.
The Adjusted $R^2$ is attempting to account for statistical shrinkage. Models with tons of predictors tend to perform better in sample than when tested out of sample. The adjusted $R^2$ "penalizes" you for adding the extra predictor variables that don't improve the existing model. It can be helpful in model selection. Adjusted $R^2$ will equal $R^2$ for one predictor variable. As you add variables, it will be smaller than $R^2$.
Sycorax
- 76,417
- 20
- 189
- 313
Eric Peterson
- 2,323
- 14
- 20
-
It is not clear, how the adjusted R square achieves the pointed properties. That is, what is the formula and how it cause the properties? – Spock77 Oct 05 '15 at 01:21
-
Adj R^2 = 1 - ((n -1)/(n - k -1))(1 - R^2) – mountainclimber11 Oct 13 '17 at 03:53
-
Where k = # of independent variables, n = # observations – mountainclimber11 Oct 13 '17 at 03:55
-
*attempting to account for statistical shrinkage* - perhaps for overfitting? – Richard Hardy Apr 24 '18 at 08:27
-1
R^2 explains the proportion of the variation in your dependent variable (Y) explained by your independent variables (X) for a linear regression model.
While adjusted R^2 says the proportion of the variation in your dependent variable (Y) explained by more than 1 independent variables (X) for a linear regression model.
-
1The distinction you are making between "independent variables" and "more than 1 independent variables" is not clear. Also, quoting Andy from below, "You don't really add new information to what was provided before." – amoeba Feb 14 '15 at 23:10
-2
R-Squared increases even when you add variables which are not related to the dependent variable, but adjusted R-Squared take care of that as it decreases whenever you add variables that are not related to the dependent variable, thus after taking care it is likely to decrease.
Glen_b
- 257,508
- 32
- 553
- 939
-
3Given that this question already has an accepted answer, this should be more of a comment. You don't really add new information to what was provided before. – Andy Feb 08 '14 at 17:44