9

The most common distinction I've seen made between interpolation and extrapolation is that interpolation is within the range of the data, whereas extrapolation is outside the range of the data. This makes sense in one dimension, but in higher dimensions I don't think it works as well. Consider the the following parameter space:

Randomly generated parameter space with empty top right corner.

Let's call the function I want to estimate $f(x, y)$. If I want to estimate the value of $f$ at the red dot in the parameter space, is this interpolation or extrapolation?

Using the "range of the data" definition it's interpolation. But the sort of problem described in this answer can still occur here. Data does not exist on all sides of the point in some sense.

Are there more precise definitions of interpolation and extrapolation that don't have this issue?

Ben Trettel
  • 191
  • 3
  • 1
    The question I ask myself is whether the point lies within a convex hull of the observed data (in parameter space). It makes it easy to see why prediction in (intrinsically) high dimensions can be so difficult! As the dimension increases the volume of the convex hull decreases -- I'd guess (though I'd have to figure out if this is actually true) that in the limit the measure of the convex hull is 0. I don't know of a formal definition, though. – David Kozak Aug 15 '18 at 01:11
  • I'm not familiar with the phrase "convex hull". Thanks for the comment! Why does the volume decrease as the dimension increases? – Ben Trettel Aug 15 '18 at 01:21
  • 1
    Does it matter whether you call it interpolation or extrapolation? I'm not saying it doesn't matter, but it's not obvious to me that it does. – The Laconic Aug 15 '18 at 01:24
  • 1
    My current argument that interpolation is better justified (which may be crazy): I expect functions to be smooth, so I have a vague prior on the Dirichlet energy of the function ($\varpropto 1/E$). This means that the most likely function satisfies Laplace's equation (minimum Dirichlet energy). But Laplace's equation has no unique solution when only one BC is available, i.e., when extrapolating. So any behavior is "likely" when extrapolating and the uncertainty is higher. Now, I could figure out definitions consistent with this idea, but rather than reinvent the wheel I figure I'd ask. – Ben Trettel Aug 15 '18 at 01:53
  • 1
    [This comment](https://stats.stackexchange.com/questions/221379/what-are-the-theoretical-reasons-for-why-extrapolation-less-reliable-than-inte?rq=1#comment418825_221380) has a criticism of the convex hull idea. – Ben Trettel Aug 15 '18 at 01:56
  • Found this after searching for something better than a convex hull: https://gis.stackexchange.com/questions/1200/what-are-definition-algorithms-and-practical-solutions-for-concave-hull – Ben Trettel Aug 15 '18 at 04:12
  • I don't think there's a common definition of the data range in multiple dimensions – Aksakal Aug 15 '18 at 19:50

1 Answers1

1

You could define the convex hull spanning those points as the range. If the point you want to predict is outside of the convex hull, you would call it extrapolation.

But what does it matter how it is called?

David Ernst
  • 2,799
  • 8
  • 14