How to interpret a negative adjusted R-squared

Question

How do I interpret a linear multiple regression with a low positive R squared and a negative adjusted R squared?

Answer 1

If you insert a constant in your linear regression $0\leq R^2 \leq 1$.

Moreover is possible to show that $R^2$ increase always, at worst remain equal, if you add one regressor. So if you ad and add regressors the $R^2$, the goodness of fit, improve regardless the meaning of the regressor that you include and regression that you achieve. This is a well known fact in linear regression.

$R^2_A$ is another goodness of fit measure that was proposed exactly for penalize the insert of “unrelevant” regressors. Is possible to shown that following the same “adding strategy” above the $R^2_A$ increase only if the t-stat associated to the parameter of the added regressor is $\geq 1$ in absolute value, otherwise it decrease. In this sense some regressors can be unrelevant and for this reason $R^2_A$ can be negative.

If your regression show low $R^2$ and negative $R^2_A$ you have a poor fit. This happen because it contain some unrelevant regressors and, probably, no many so relevant. However you have to note that the goodness of fit usually is not the key characteristic of your regression (see below).

Warning: from this story we can think that maximization of $R^2_A$ is a good strategy. In some sense it is infact some stepwise regression tools follow this rule also. However we have to note that this strategy can be useful, probably not the best, only for variable selection in predictive oriented regression. In regression for causal inference, statistics like $R^2_A$ says us almost nothing about the quality of our models. Moreover the $R^2$ is very interesting in algebraic and geometric sense but says us almost nothing in both prediction and causal inference, the two main scope of regression in practice.

Answer 2

Keep in mind that adjusted $R^2$ is a model selection criterion, so neither its sign nor its magnitude have any statistical meaning. Low adjusted $R^2$, whether negative or not, just tells you that your model is a poor fit.

EDIT (because people seem to forget that "$R^{2}$" and "adjusted $R^{2}$" aren't the same thing): "adjusted $R^{2}$" unfortunately is a poor choice of name for a model selection criterion, because it makes it sound like it is a somehow "better" version of the $R^{2}$, which is the squared correlation of your regressand and regressors.

But as I wrote above, adjusted $R^{2}$ is a model selection criterion, and like all of those it tries to counter-weight "model fit" (=actual $R^{2}$, where higher means better) with "model size" (=number of parameters, where higher means worse). Without context, i.e. a second value of adjusted $R^{2}$ from another model, it has no meaning. It certainly does not have any statistical meaning like $R^{2}$ has.

Theil's adjusted $R^{2}$ kicked off the development of more sophisticated model selection criteria, like Mallow's $C_p$ and Akaike's "information critierion." Those, too, have "meaning" only in comparison, not in absolute value.