How to interpret redundancy?

Question

I have trouble making sense (i.e. real-world sense…) out of some of my results.

I have Y and X1 and X2 for different geographic areas. Meaning they are the same variables, but their actual values are different per area.

I did a multiple linear regression and the R-squared is good for all of them (F-test indicates significance).

I then calculated the semi-partial (or part) correlation to figure out the relative importance of X1 and X2 in explaining variance of Y. As I understand it, adding these two correlation coefficients squared, will add up to the overall R-squared - and if not, that "remainder" is variance redundantly predicted by X1 and X2.

For my 7 models, this redundancy varies from very little, to a lot: can anyone give me insights as to what that could imply about my X1, X2 and Y?

There's a lot of stuff out there on suppression (when the squared semi-partial correlation added together is larger than the overall R-squared), but I have not yet come across anything really useful about redundancy.

[I fear this might be a very basic statistical question, I apologize - but it makes me wonder why I can't find anything informative on the WWW!]

Answer 1

I'll try to answer your question with what I hope is a common sense example.

The partial $R^2$ of a given predictor represents the unique variance in your dependent variable the predictor is able to account for above and beyond the that of the other variables. There are some situations where we would imagine there to be no redundancy in our predictors. For example if we were to fit the following model:

$${\rm Math\ Ability}_i = \beta_0 + \beta_1{\rm Age}_i + \beta_2{\rm Gender}_i + \varepsilon_i$$

We would expect very little redundancy in our predictors because they are unrelated (gender does not depend on age) and would each explain unique variance in math ability.

There are other situations where one predictor, despite being related to the outcome variable, may provide very little unique information about the outcome when included in a model with more meaningful predictors. For example:

$${\rm Math\ Ability}_i = \beta_0 + \beta_1{\rm Age}_i + \beta_2{\rm Height}_i + \varepsilon_i$$

In this scenario, we would expect height to be highly redundant with age, and, though age might still have a substantial partial $R^2$, we'd expect a small value for height. Although it is certain that height and math ability are related (adults tend to be better at math than small children), almost all the variance in math ability explained by height is also explained by age (which likely explains a more substantial amount of variance). The unique contributions of age beyond height are noteworthy, while the unique contributions of height beyond age are negligible.

In regards to your data, it is difficult to speculate as to why you may or may not have redundancy in your predictors (if I had named the variables $Y$, $X_1$, $X_2$ and $X_3$ in the examples above, I doubt my reasoning would have been as clear).