Removing the effect of unknown independent variables

Question

I have 25 years of crop yield data (Y) and Minimum Temp, Max temp, and Rainfall as X1, X2, and X3. When I do regression analysis I get a result.

My question is that for the crop yield (Y), in addition to the X1, X2, and X3 independent variables, there are many other factors which can influence crop yield, such as good quality of soil, seed, fertilizers, etc. Since I have no data for all these variables, how can I remove the effects of these variables and look only at the effects of X1, X2, and X3 on crop yield (Y)?

If you don't have data for it you can't really remove their effect, since you don't know what effect they have. — user2974951, Jul 10 '19 at 05:57
Yes but the Value of Y is due to other IVs also including X1,X2 and X3. So how to deal that? — Nirmal Mani Dahal, Jul 10 '19 at 08:18

IrishStat · Answer 1 · 2019-07-10T15:17:32.193

The errors from your model may reflect omitted variables. The omitted variables can either be stochastic or deterministic. The effect of the unknown/latent and waiting to be discovered deterministic variables (reflecting omitted deterministic series) can be level/step shifts, local time trends, seasonal pulses, monthly effects and/or simply pulses. After adjusting for these effects by Intervention Detection / Model schemes http://docplayer.net/12080848-Outliers-level-shifts-and-variance-changes-in-time-series.html augmentation one can then study any remaining stochastic effects by examining the auto-correlation of the model residuals and the cross-correlation of model residuals with lags of the user-specified predictors to suggest appropriate model augmentation reflecting possible omitted stochastic series.

Find correlation between two time series. Theory and practice (R) is an interesting thread that might be of help to you.

Removing the effect of unknown independent variables

1 Answers1