1

I am performing a regression on a large dataset that is fairly noisy. The line I am running in R is:

lmfit <- lm(predictVariable ~ dataSet[,1:10])

so I have 10 endogenous variables. The results are somewhat strange:

  1. significant t-statistics for all the coefficients

  2. $R^2 = 0.25$

But when I run a regression again on the fitted and actual variables, I get this:

run1 <- lm(predictVariable ~ lmfit$fitted.values)

The coefficient in front of fitted values is 1.0001 and the intercept is ~0. But now when i run it in the reverse way:

run1 <- lm(lmfit$fitted.values ~ predictVariable)

The coefficient is now 0.26 and the intercept is also ~0. Why are the coefficients in the last two regressions so different? Does this mean my endogenous variables are colinear, or that my fit is poor? When I plot the fitted.values versus the actual, I see that there are many actual values close to 0 but the fitted value is higher and lower. Can anyone help me understand these regression results?

chl
  • 50,972
  • 18
  • 205
  • 364
Sam
  • 11
  • 2
  • 1
    One thing to realize about linear regression is that regressing $Y$ on $X$ is *not* the same as regressing $X$ on $Y$, see [here](http://stats.stackexchange.com/questions/22718/regressing-x-on-y-or-y-on-x/22721#22721). – gung - Reinstate Monica Jun 08 '12 at 02:43
  • You shouldn't expect a simple relationship between $E(Y|X)$, which is estimated by the first model, and $E(X|Y)$, which is estimated by the second model, as @gung has pointed out. Also, can you please clarify what you mean by an "endogenous variable"? From the context it appears you mean it to refer to the predictor variables but I (and perhaps others) am not very familiar with that terminology. – Macro Jun 08 '12 at 02:46
  • I agree with gung and Macro. The two regressions are different problems. Also the slopes should of course be different. If Y=aX then X=Y/a. – Michael R. Chernick Jun 08 '12 at 09:59
  • Can you post the plots. – simmmons Jun 08 '12 at 13:15
  • Yes--gung your explanation makes a lot of sense. If I do a total least squares, then should the fitted coefficients be the the inverse of each other? – Sam Jun 10 '12 at 13:37

0 Answers0