4

I want to use forward selection to choose predictors in a multiple linear regression model. If you have a regression with N predictors and want to add another predictor, is there a way to update the coefficient estimates that is more efficient than just estimating a new regression from scratch?

Karolis Koncevičius
  • 4,282
  • 7
  • 30
  • 47
Fortranner
  • 586
  • 2
  • 12
  • In `R`, the `update` function performs such a job, although I do not know if it does so with special regard for efficiency. In any case, you may want to inspect that function's code. – Christoph Hanck Jun 15 '18 at 14:43
  • The `update` function do not even try to be fast, it just updates the model formula and reruns. – kjetil b halvorsen Jun 15 '18 at 15:15
  • You should not use forward selection. It may help you to read my answer here: [Algorithms for automatic model selection](https://stats.stackexchange.com/a/20856/7290). – gung - Reinstate Monica Jun 18 '18 at 14:28

1 Answers1

1

The least-squares solution to X*b = Y is given by the normal equations

X′X b = X′Y

Adding another predictor is equivalent to adding a column to X. Given the old matrix of cross-products, to get the new X'X you only need to calculate the dot product of the new column with the old columns. This saves time compared to recomputing X'X from scratch.

Fortranner
  • 586
  • 2
  • 12