6

I am trying to implement all-possible regressions in order to select the best predictors of stock returns from an exhaustive list of potential economic/fundamental variables.

My response variable y (i.e. stock returns) is a panel of 3000 securities (cross-section), each having 384 observations (time-series).

Would anyone please suggest the best way to handle this procedure in R, in the context of panel data? I came across the package leaps, but it addresses the case of y as a response vector rather than a response matrix.

Thank you very much,

Mayou
  • 737
  • 3
  • 12
  • 29
  • 1
    I don't recommend this approach. But to answer your question, you can create a list of all possible of variables using something like `expand.grid()` to create an matrix for which the columns correspond to variables and the cells correspond to whether the variable is included in that model (there is probably a better way) and then fit each model (row). – Ellis Valentiner Aug 23 '13 at 16:22
  • @EllisValentiner Thank you for getting back to me. I might not have been very clear actually. Let's take an example: one of my predictors is Inflation. Inflation has a time-series of values, but cross-sectionally, it is constant for all securities (since it is a macro-variable, and not a **firm characteristic**). Therefore, I end up with 3000 securities, each having a monthly time series of returns (dependent variable). On the other hand, I have a list of predictors (inflation, oil prices, ...) that each have a time-series of values, but each value is constant for the cross-section. – Mayou Aug 23 '13 at 16:25
  • How would I do `leaps()` in this case? Should I do it separately for each security? I don't think so as this would exclude any cross-sectional dynamics... Therefore, what is the solution? Is there anyway of doing it simultaneously for the whole panel (time-series/cross-section)? – Mayou Aug 23 '13 at 16:29
  • 3
    Rather than highly problematic stepwise regression it would be better to use a statistically principled approach as discussed at length in stackexchange. – Frank Harrell Aug 23 '13 at 16:47
  • @FrankHarrell Which statistical procedure are you referring to? Would you please point me the right thread on Stack Exchange? Thank you very much. – Mayou Aug 23 '13 at 17:19
  • 2
    There are many, many threads on this. Think about data reduction (blinded to $Y$) or penalization (L2 - ridge regression, L1 - lasso, combination = elastic net). You might also entertain random forests. – Frank Harrell Aug 23 '13 at 17:39
  • 3
    "All possible regressions" *will not* allow you to "select the best possible predictors". If this doesn't make sense / you want to know why, it may help to read my answer here: [algorithms-for-automatic-model-selection](http://stats.stackexchange.com/questions/20836//20856#20856). There are some approaches that can be used w/ situations like yours. These include penalized methods (ridge regression, LASSO, LARS, elastic net) & cross-validation (try reading some of the highest voted threads under the [tag:cross-validation] tag). – gung - Reinstate Monica Aug 23 '13 at 17:42
  • @FrankHarrell Very helpful, thank you. I have just started reading about LASSO, and still at early stages of understand how it works, and how it could be implemented in R. – Mayou Aug 23 '13 at 17:43
  • @gung Thank you for the explanation. I am new to this world of "model selection" and still confused by the different methodologies. Your comment is very helpful, thanks. – Mayou Aug 23 '13 at 17:44
  • Your two previous questions are essentially duplicates of this one and contain essentially the same material: http://stats.stackexchange.com/questions/68119/forward-stepwise-regression-procedure-application-to-panel-data and http://stats.stackexchange.com/questions/68165/could-stepwise-regression-lead-to-a-final-model-with-no-statistically-significan. **It is more constructive to ask the *single* question "How could I go about achieving this objective" rather than asking a series of questions about "Would such-and-such achieve my objective."** – whuber Aug 23 '13 at 17:51
  • @FrankHarrell Would LASSO be applicable in the context of panel data (response variable is a panel of 3000 securities with 384 observations each) ? – Mayou Aug 23 '13 at 17:55

2 Answers2

4

After re-reading your question, I believe you mean to ask about model selection among your candidate predictor variables, and not actually running all possible regressions. Fitting all possible models from a given set of predictors is subject to a high degree of data-mining bias. Since many such sub-models will be highly correlated with each other (because they include almost entirely the same set of factors) you would need to adjust your t-statistics to account for the probability that, among the entire set of correlated models, some models just randomly look successful within the particular sample data you have. Adjusting for so many models would imply that you'd need an unrealistically high t-statistic to have any confidence in coefficients from the model that you finally select.

Some better approaches might be Bayesian linear regression where you specify what prior distribution you think is realistic for the coefficient on each of the predictors, or regularized regression like Lasso or Ridge, where you impose some penalty term for how dense or big the set of estimated coefficients is (e.g. the fitting procedure will try to favor models with fewer terms in a suitable sense).

If you start out from one of these perspectives, then there is less risk in testing a couple of models that you think have strong prior evidence.

But in general, if you simply look at all n-choose-k subsets of factors, for k = 1 through n, then by simple random chance, some model will appear very strong but not due to actual forecast efficacy. You should avoid this.

ely
  • 2,272
  • 18
  • 31
  • Thank you for your detailed answer, very much appreciated. I am in the process of reading about LASSO, its intuition, and implementation in R. I am very new to the field of "model/variable selection" and still confused about the different methods. – Mayou Aug 23 '13 at 17:46
0

I suggest you to use usual panel data analysis approach [in R this involves the use of plm package). There are three categories of explanatory variables (observed or unobserved) that this approach takes into account. First, the variable which is same across each stock but varies over time (economic fundamentals), second is the variable which varies across each stock but doesn't change over the time (e.g., management style), and third includes the variable which varies over time and also across stock (firm's earnings). If the first two variables are unobserved, they are taken into account by using two-way fixed effects (stock effects and year effects), thus use of panel data avoids omitted variable bias arising from the exclusion of these two categories of variables. So, the only bias that arises from the exclusion is due to the omission of last category of variable. If you are sure that your model includes all that belonging to last category, then there is no omitted variable bias. The significance of these variables indicate that they might be important in influencing the stock returns. That being said, which approach to use depends on the purpose of your research.

Metrics
  • 2,526
  • 2
  • 19
  • 31
  • Thank you very much for your response. `plm` package indeed is great for dealing with large datasets. The purpose of my research (I am a practitioner, not an academic), is to develop a stock selection model in the form of an economic factor model. This will support our current stock selection process in place. I have gathered data for 14 possible predictors (based on economic intuition and some backtesting procedures). However, I need to reduce the dimensionality of my variable space, and select only predictors with a "non-zero" premium. – Mayou Aug 23 '13 at 17:29
  • In other words, I need to select the variables that help predict the stock returns, and for which the risk is rewarded by the market. Which "saine" statistical procedure would you recommend in this case? – Mayou Aug 23 '13 at 17:30
  • @FrankHarrell I have tried to apply the **Fama-MacBeth two-pass regression** to discover the predictors that have a statistically significant premium. However, the method is very "questionable" (due to errors-in-variables problem) and leads to very weak results. Any suggestions for an alternative procedure for variable selection? Thank you. – Mayou Aug 23 '13 at 17:32
  • 1
    I think you can use factor analysis or principle component analysis for the selection of variables. – Metrics Aug 23 '13 at 17:32
  • Thanks for your suggestion. However, the problem with PCA is that the components don't have any "economic" meaning. Also, the model will be used later to predict stock returns. I can't think of a proper/meaningful way of forecasting the values of the principal components. – Mayou Aug 23 '13 at 17:38
  • I haven't yet used these, but I thought you might want to look at. There is whole stuff in Machine Learning literature that deals with data mining literature if your goal is not academic. But, it seems paradoxical since you are comparing this with academic model. – Metrics Aug 23 '13 at 17:56
  • For a practitioner, it is somewhat hard to draw the line between "practicality" and "theory". I need to develop a model using sound theoretical statistical methods, while having variables with sound economic intuition. – Mayou Aug 23 '13 at 18:03
  • I understand that. But, the point is you choose the variables based on theory and then see whether they are statistically significant in statistical model; not the opposite. Even if you include economic factors, your model will be still biased because there is a new literature in finance (behavioral finance) that argues that non-economic factors should be also incorporated in the asset pricing model, e.g., weather which influences the mood of the investor and so does the trade. See the paper by Hirshleifer and [Shumway](http://www.jstor.org/stable/3094570) for that. – Metrics Aug 23 '13 at 18:14
  • 1
    You are right indeed. But when I refer to "Economic factor model", this does not mean that the model only incorporates economic factors. The variables that I am testing fall indeed under several categories: fundamental, technical, economic and behavioral. – Mayou Aug 23 '13 at 18:16
  • I have not heard of Fama-Macbeth. From its description as "two-pass" it is questionable. – Frank Harrell Aug 23 '13 at 18:43