4

Question

The question is: how to smooth out kinks in individual demand curves in a GBDT model without underfitting on the price variable?

Background

We have some GBDTs demand models already in place (with monotonic constraints that ensure non-increasing aggregate and individual-client demand curves). The problem with these algos is that while the aggregate demand curve (averaged over all user profiles) is smooth, individual demand curves are highly irregular, non-smooth, exhibiting sudden changes that could be potentially exploited by the clients and/or their advisors.

Ideas

One idea we had was to fit two models and use first of them as an offset in the second one. First one with all price-related variables, and second - without them, and only the second one using a high-accuracy non-linear (tree-based) algo. The only algo that met the smoothness requirements for the first-stage model was GLM (strictly GLM, as more general Elastic Net only worsened accuracy due to tree-based feature selection process we used), which is obviously bad for accuracy (despite the second stage model being non-linear) and results in both lower (absolute) elasticity of demand and increased implementation complexity.

Any other ideas how to solve the problem? Can it be achieved using a single tree-based model? Ideally if the final algo was a "non-academic" (battle-tested) one, like xgboost or lightgbm, for production support reasons.

mirekphd
  • 153
  • 6

1 Answers1

3

GBTs are a variation on a tree model. One downside to trees is exactly what you've found -- because tree induction works by splitting on features, then handling the children separately, the results can be highly discontinuous. The "left" and "right" child leaves at each can be totally different.

By contrast, some models are explicitly designed to accommodate continuity in the prediction functions. I don't know whether or not this works for your case, but an example is cubic spline regression, which explicitly enforces continuous first and second derivatives at the knots (between knots, the function is a polynomial, i.e. smooth).

There are even monotonic spline models.

  • Randall L. Dougherty, Alan S. Edelman and James M. Hyman. "Nonnegativity-, monotonicity-, or convexity-preserving cubic and quintic Hermite interpolation" Journal: Math. Comp. 52 (1989), 471-494

  • F. N. Fritsch and R. E. Carlson. "Monotone Piecewise Cubic Interpolation" SIAM Journal on Numerical Analysis (1980) 17:2, 238-246

Sycorax
  • 76,417
  • 20
  • 189
  • 313
  • Perfect! There's even a `scipy` implementation of piecewise cubic interpolation called `scipy.interpolate.PchipInterpolator` I wish that monotonic constraints were implemented that way in these select few "production-ready" algos we are required to use... – mirekphd Jun 11 '21 at 15:19
  • 1
    It’s in my TODO list … – Sycorax Jun 11 '21 at 15:19