Regression to Classification and back to Regression

Question

Is it reasonable to transform regression problem into classification by binning target variable into classes and construct regression curve separately on each class?\

Precisely, if my goal is to solve regression problem are the following steps reasonable:

1. If my target variable is $Y$, Create $m$ classes $Y_1 =\{Y:Y<y_1\}, Y_2=\{Y:y_1\leq Y<y_2\},\dots,Y_m=\{Y:Y\geq y_m\}$.
2. Construct classifier $p_j(x)=Pr(Y \in Y_j|X=x)$.
3. Construct regression curves for each class separately $E[Y|Y_j,X=x]=f_j(x)$.
4. Estimate final regression curve by $E[Y|X=x]=\sum_{i=1}^m p_j(x)f_j(x).$

Theoretically, if our goal is to construct regression curve $E[Y|X=x]$ than from identity $$ E[Y|X=x]= \sum_{i=1}^m Pr(Y \in Y_j|X=x) E[Y|Y_j,X=x]=\sum_{i=1}^m p_j(x)f_j(x)$$ it looks like steps described above are just a waste of time. But it’s possible that the estimation of $ p_j(x)f_j(x)$ could be done more effectively. Any literature or comment would be helpful.

Answer 1

Short answer: It is most likely not reasonable.

While the question lacks details of the actual goal of the analysis, my best guess is that you assume your response to be nonlinear, since the procedure you describe roughly corresponds to replacing linear response with a piecewise linear response.

If you indeed need to do this, then you could benefit from some non-parametric regression, for example Gaussian process regression or regression splines that directly model non-linear responses and would likely provide both more flexibility and better guarantees than you proposed approach.

If you really need the response to be piecewise linear, you could likely construct a model that would fit both the regressions AND the locations of the breakpoints at the same time (at least this should be feasible with Stan, not sure about the frequentist case).