How can I efficiently calculate an OLS fit for N multiple variables for a rolling window?
I've worked out how to do it for 1 and 2 variable linear fits, I'd like to extend to the general case of N variables if possible (or at least to 3). I don't want to calculate the whole regression for each individual window, I want to update my calculation with each new data point.
For example, for 1 variable fit of the form: $y_i = \alpha + \beta x_i + \epsilon_i$ (where $\epsilon$ is the error term)
$$\beta = (n\sum_{i=1}^{n} x_i y_i - \sum_{i=1}^{n} x_i \sum_{i=1}^{n} y_i) / (n \sum_{i=1}^{n} x_i^2 - (\sum_{i=1}^{n} x_i)^2)$$
$$\alpha = (\sum_{i=1}^{n} y_i - \beta \sum_{i=1}^{n} x_i)/n$$
In this case I only need to maintain rolling values of $\sum x$, $\sum y$, $\sum xy$, and $\sum x^2$ and I can easily calculate the fit parameters quickly at each window step without redoing the whole thing.
I achieved the same for a 2 variable multiple fit of the form: $y_i = \alpha + \beta_{1} x_{i1} + \beta_{2} x_{i2} + \epsilon_i$
It's pretty messy: $$\sum X_1Y = \sum_{i=1}^{n} x_{i1}y_i - \frac{1}{n}(\sum_{i=1}^{n}x_{i1} \sum_{i=1}^{n} y_i)$$ $$\sum X_2Y = \sum_{i=1}^{n} x_{i2}y_i - \frac{1}{n}(\sum_{i=1}^{n}x_{i2} \sum_{i=1}^{n} y_i)$$ $$\sum X_1X_2 = \sum_{i=1}^{n} x_{i1}x_{i2} - \frac{1}{n}(\sum_{i=1}^{n}x_{i1} \sum_{i=1}^{n} x_{i2})$$ $$\sum X_1X_1 = \sum_{i=1}^{n} x_{i1}^2 - \frac{1}{n}(\sum_{i=1}^{n}x_{i1})^2$$ $$\sum X_2X_2 = \sum_{i=1}^{n} x_{i2}^2 - \frac{1}{n}(\sum_{i=1}^{n}x_{i2})^2$$
we then get $$d = (\sum X_1X_1 \sum X_2X_2) - (\sum X_1X_2)^2$$ $$\beta_1 = \frac{1}{d}(\sum X_2X_2 \sum X_1Y - \sum X_1X_2 \sum X_2Y)$$ $$\beta_2 = \frac{1}{d}(\sum X_1X_1 \sum X_2Y - \sum X_1X_2 \sum X_1Y)$$
Again, here I only need to maintain rolling sums of $\sum x_1$, $\sum x_2$, $\sum y$, $\sum x_1y$, $\sum x_2y$, $\sum x_1x_2$, $\sum x_1^2$, and $\sum x_2^2$ to calculate the regression, I don't need to do the fit individually for each window, and so it's really quick. (Note I only care about the $\beta$ values).
I now want to extend this to 3 variables (and possibly more). I've started working through the algebra and it's hairy. I feel sure I must be missing a trick or a pattern here that makes the extension to 3+ variables relatively easy.
Can anyone suggest how I can extend my technique above to 3 variables (or possibly more) or otherwise to efficiently calculate the $\beta$ for a rolling window? Thanks!
