1

This is something that bothers me for quite some time, but I didn't find yet a satisfactory answer. I hope that the wisdom of the people hear will help me to clarify this:

In a multivariate regression, right-side variables can be included to control for the contribution of the other variables. For example, in a regression that predicts, let's say, math abilities by shoe-size, we might get a strong relationship (because younger children tend to have smaller feet and less mathematical knowledge). But when including the covariate "age" in this regresion, the relationship between math abilities and shoe-size would disappear.

Thus, the regression estimates provide the partial correlation of the predictor with the dependent variable (that is, the estimate captures the contribution of that predictor beyond the contribution of all the other predictors).

However, by including the control variable of "age" in our regression - aren't we creating a problem of collinearity? As age is correlated with shoe-size (otherwise, why would we want to control for it?). Isn't that a problem for obtaining accurate estimates?

Galit
  • 107
  • 1
  • 9
  • It isn't a problem for accurate estimation, but it is a problem for uncertainty estimation and *inference*. Basically, l if your goal is causal inference, 'controlling' through multiple regression isn't going to save you from a poorly specified model. – mkt May 06 '19 at 09:34
  • @mkt I need some more clarification - when is it OK that one predictor correlates with another for the purpose of control? If I'm interested in the unique contribution (cleaning out the control variable) - shouldn't it be always OK, despite collinearity? – Galit May 06 '19 at 11:51
  • I don't have the time to get into this at the moment, but here is some useful discussion about how 'controlling' for a variable is not as easy to achieve as people seem to think: https://statmodeling.stat.columbia.edu/2019/01/25/regression-matching-weighting-whatever-dont-say-control-say-adjust/ , https://statmodeling.stat.columbia.edu/2006/04/28/amusing_example/ , https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4816570/ – mkt May 06 '19 at 12:26
  • 1
    I appreciate your time, so if you don't have time I hope someone else reads this and can reply. I'll try to phrase it differently: If I have two predictors that correlate with each other, then theoretically a multiple regression should give the contribution of each to the outcome, after removing their shared variability (I'm leaving aside issues of "control" or not). If that is so, why is collinearity ever a problem? Do we never expect this shared-variability to be non-zero (then, why removing it?)? If it is a problem, why is partial-correlation ever legit? – Galit May 07 '19 at 12:24
  • Have you seen the answers for this question? https://stats.stackexchange.com/questions/1149/is-there-an-intuitive-explanation-why-multicollinearity-is-a-problem-in-linear-r . They may address at least part of your question. – mkt May 07 '19 at 13:55
  • These are also relevant: https://stats.stackexchange.com/questions/168622/why-is-multicollinearity-not-checked-in-modern-statistics-machine-learning & https://stats.stackexchange.com/questions/86269/what-is-the-effect-of-having-correlated-predictors-in-a-multiple-regression-mode – mkt May 07 '19 at 13:56

0 Answers0