What's the correct term for regression where you first regress on one input variable (feature), take the errors, regress on the next feature, etc.? In what specific cases is this useful? Are there any references for this technique?
More specifically, I am talking about regressing $y$ on $x_1$, then regressing $(y-b_1\times x_1)$ on $x_2$, etc. This is because each $x$ is a time series, and needs to be sampled differently, so I cannot take them all as inputs to the regression at the same time.
Taking some liberties, you are probably talking about the instrumental variable techniques, or IV. They are used to help argue causality and to control for endogenous variables. Commonly part of econometrics.
Alternatively, you may be thinking of the FWL theorem
Are you talking about 'step-wise regression'? However, in step-wise regression, we regress response on one variable, then we regress response on the new selected variable as well as the previous variable, i.e. regress response on two variables. Next, we regress response on three variables and so on.
