2

I understand that correlation (I'm referring to Pearson product-moment correlation coefficient) and regression share some similarities from this post: What's the difference between correlation and simple linear regression?

However, I'm wondering the limitations of multiple regression.

Scenario: I would like to explore the associations between a number of continuous variables (e.g., 10 variables). Of course, I would first compute a correlation matrix to observe their inter-correlations.

Next, I would like to explore the effect of the variables on the one or two DVs I’m interested in. I am wondering whether I should run many multiple regression models. I understand that even if two variables (X1,X2) have a significant correlation with another variable (Y), entering X1 into the regression model first might make X2 insignificant. It means that X2 being insignificant in the model doesn't mean X2 is useless. Therefore, my question is: in general, why do we make a regression model even if our correlation matrix could address some of my questions?

JetLag
  • 1,105
  • 2
  • 9
  • 22
  • 1
    If you are interested in multiple regressions of every variable on the other, you may want to model *partial correlations* instead, using a graphical model (e.g. graphical LASSO or ridge). This has the interpretation of regressing every variable on every other and results in a conditional independence network. You may also want to be careful with the use of the word 'significant' in this context, because of the combinatorial explosion when interested in the effect of everything on everything. I doubt any significance remains after multiple testing correction. – Frans Rodenburg Nov 01 '17 at 09:26
  • It seems you are trying to estimate a (causal) Bayesian network (you want to know how each variable affects each other as a system, you do not want only one "outcome regression"), is that it? – Carlos Cinelli Nov 20 '17 at 22:14
  • @CarlosCinelli Actually there are two DVs that I’m interested in, and my goal is to see how other variables would influence each of them. – JetLag Nov 21 '17 at 00:19
  • Could you draw a graph of the relationships you're thinking? For example, some models that might explain your description in your question is something like $x_2 \rightarrow x_1 \rightarrow y$ or $x_2 \leftarrow x_1 \rightarrow y$. – Carlos Cinelli Nov 21 '17 at 00:26
  • @CarlosCinelli Technically, the ultimate goal of my analysis is to see if Z1, Z2, Z3, Z4, ..., Z10 would moderated the relationship between X and Y. – JetLag Nov 21 '17 at 00:33
  • @JetLag causally? – Carlos Cinelli Nov 21 '17 at 00:35
  • @CarlosCinelli observationally. – JetLag Nov 21 '17 at 00:39
  • If you are considering a multivariate normal distribution, you can just regress $Y$ on $X$ and $Z_1 \dots Z_{10}$ with the appropriate interactions. – Carlos Cinelli Nov 21 '17 at 00:40
  • @CarlosCinelli Thank you very much. If I have 10 potential moderators, then I need to run 10 regression models? – JetLag Nov 21 '17 at 00:42
  • No, but maybe you're no being clear about what you actually want to do. You should take a look at graphical models, specially bayesian networks. By your questions it feels you want to make inferences about the structure of your problem not just an outcome equation. – Carlos Cinelli Nov 21 '17 at 00:56
  • @CarlosCinelli I understand your point of regressing Y on X and Z1 ... Z10 now. I was mistaken because I couldn't read the notation on the phone. I will take a look at bayesian networks. Thanks. – JetLag Nov 21 '17 at 01:01

0 Answers0