3

I determined the hierarchy of a small herd of milking cows. It is obvious that the rank of a given cow does highly depend on her age. Not as substantial as the age, but probably still quite meaningful is her weight. Other variables that might explain her rank to some extent (as number of calves she gave birth to, sacral height etc.) are available as well.

I can determine how much (variance) of the hierarchy is explained by any one of these variables when calculating the R² value of the respective bivariate linear model.
But I can not simply sum the R² values up, since they yield much more than 100 % in total. I guess this is, because the explanatory variables are correlated themselves as well. I can neither sum them all up and scale the results down to 100 % in total, since they don't explain 100 % of the hierarchy (a multiple linear regression model with all of the possibly explaining variables has an (multiple) R² value of almost 0.9, an adjusted R² value of almost 0.86, respectively). Scaling down to 90 % wouldn't work either, if I'm not mistaken, because the multiple linear regression model does not (necessarily) weigh the single explanatory variables equally. Furthermore I don't see the possibly explanatory variables do indeed all explain something.

How would one determine, which explanatory variable does contribute how much on it's own?

Michael R. Chernick
  • 39,640
  • 28
  • 74
  • 143
mattu
  • 43
  • 7

2 Answers2

6

The important thing is that the influence that each predictor has depends on which other predictors are in the model. So the effect of each predictor depends on the model you're using, and which other variables are in it. For example, you might have two highly collinear variables that both have a large $R^2$ when used in a bivariate linear model, but, because of the collinearity, that doesn't mean that you need them both in your model, or that your model will be better if both variables are in it.

So start by using methods to address collinearity among your predictors and find a plausible model with several variables. Once you do have a model, you can assess which predictor variables are more or less important to predicting the response by using added variable plots. If there's a strong linear relationship in a plot, that variable is important to the model when it already contains the other variables.

What does an Added Variable Plot (Partial Regression Plot) explain in a multiple regression?

Nick Koprowicz
  • 838
  • 3
  • 12
  • +1, especially for the initial remark, which gets to the heart of the problem. Welcome to our site, Nick! – whuber Dec 28 '19 at 21:59
  • Upvoted for the clarity and excellence of the answer. – James Phillips Dec 28 '19 at 23:22
  • The influence of each predictor also depends on the experimental design. For example, if the design matrix is orthogonal, the effect of each predictor does not depend on the other predictors in the model. – JTH Dec 31 '19 at 11:49
2

For the question: "How much of a dependent variable is explained by each of a bunch of independent variables?", the short answer is the so-called adjusted R-squared.

It is a modified version of R-squared that has been adjusted for the number of predictors (degrees of freedom) in the model. The adjusted R-squared increases only if the new variable actually improves the model more than would be expected by chance alone. In fact, it can actually decrease when a predictor fit improvement to the model is less than expected by chance.

AJKOER
  • 1,800
  • 1
  • 9
  • 9