6

I read a lot about polynomials and splines (and in case of the latter also lots of it derivates) and often some special cases were introduced to explain, mostly, why a spline is more suitable than a polynomial.

Nevertheless, at the moment I feel like it doesn't play a role in 99% of the cases whether I choose a spline or a polynomial. But still I have to make this decision. Is there an easy and simple advice how to proceed?

Or maybe a bit more general: Are there cases where polynomials are better than splines?

Ben
  • 2,032
  • 3
  • 14
  • 23
  • 2
    It depends on what you are trying to do of course, and what the underlying mechanism of the data generation is. Splines are piece-wise defined. Is your data piece-wise created? Splines can be heavily parameterized, $O(Mp_0), p_0 \sim 3, 4$ where $M$ is the number of pieces and $p_0$ is number of parameters per segment. Polynomials $O(x^p), p \gtrsim p_0$ only have $p+1$ parameters. If you're trying to learn/generalize w/splines, you'll need lots of data for robustness: $O(10Mp_0)$ vs. $O(10p)$ for polynomials. If you only need to interpolate/smooth/visualize 1 data set, then splines are fine. – Peter Leopold Jul 29 '19 at 13:27
  • At first thanks a lot! Maybe a dumb question but when do I know my data is piece-wise created? I have lots of separate data sets in my mind which I want to interpolate. The data is rather robust but I do not want to check each manually. All I can say up to now is that I like splines more but I feel save with polynomials here. – Ben Jul 29 '19 at 14:29
  • 3
    I wrote a long essay about this from a predictive point of view if you'd like to check it out: http://madrury.github.io/jekyll/update/statistics/2017/08/04/basis-expansions.html – Matthew Drury Jul 29 '19 at 19:07
  • 1
    The trouble with polynomials is that you have no idea what the curve is doing between data points. There is no guarantee whatsoever that a position on the curve between the data points will be near a data point. – CElliott Aug 15 '19 at 15:46

1 Answers1

8

My RMS book and course notes go into detail about this. Briefly, polynomials are too restrictive, allow a point in one part of the curve to too greatly influence the fit in other parts of the curve, and the fits are not as good as segmented polynomials (splines). Polynomials cannot well approximate threshold effects or logarithmic shapes.

Frank Harrell
  • 74,029
  • 5
  • 148
  • 322
  • Thank you, I will need some time to go through the details. Therefore in advance: There is no case where polynomials do better respectively are more advisable? – Ben Jul 29 '19 at 14:32
  • 1
    Polynomials can occasionally do better when there are so many ties in the data that you can't decide where to place the knots. – Frank Harrell Jul 30 '19 at 17:58
  • My assumption was totally the opposite. I thought, when there is a "clear line", it doesn't really matter whether polynomial or spline. But with lots of "overlapping" data points, almost not representing a structure, I thought a spline would be more useful? – Ben Jul 30 '19 at 19:05
  • 1
    Besides the difficulty knowing where to place knots in this special case, if you knew something about true extreme shape changes you could of course place a knot at every x-point of high frequency to allow for anything. – Frank Harrell Aug 02 '19 at 13:01