1

And what is the opposite? "Global" methods? What the differences are? I have found reference to "local" methods in this answer about KNN: https://stats.stackexchange.com/a/104261/107213

Marcin
  • 917
  • 1
  • 7
  • 11
  • For your information, this is not very popular distinction. Honestly, I don't recall seeing it anywhere. – Tim Dec 12 '18 at 20:04
  • It might be more productive to think of $k$-NN as a method that has no learned parameters (it just finds nearby points), while something like a logistic regression learns weights (coefficients) for features. – Sycorax Dec 12 '18 at 21:03
  • @Sycorax, you can learn optimal *k* in KNN, it's like an effective window size in kernel methods, where it turns out to be the most important characteristic, maybe more important than the type of kernel itself – Aksakal Dec 12 '18 at 21:08
  • @Aksakal You can optimize $k$, most commonly with cross-validation, but it's not amenable to estimation from the training data in the same manner that the coefficients of a logistic regression: $k$ is a hyperparameter, not a parameter. – Sycorax Dec 12 '18 at 21:11

1 Answers1

2

I don't think there's a precise definition or an exhaustive list of local methods in data science. I bet that the term local (nonlocal is the opposite) came either from physics or mathematics into what you call a data science.

In physics local means that an object is influence only by its immediate surroundings. In the old days locality was restricted to a literal collision or some sort of hard connection between objects through some kind of a medium like ether. This was before the notion of forces such as gravity, and their modern understanding where there's no need for any medium at all, the fields go through vacuum too.

So, today the meaning of locality in physics could be linked to the speed of information transfer. For instance, Earth and Moon interact still locally despite being somewhat far away. Here we mean that their movements are described by gravity force/field, and the disturbances would be passed along by speed of light, and, consequently proximity matters. Ceteris paribus, distant objects impact us less than objects in close proximity. This is very convenient, because although every object interacts with every other object in the universe through gravity force, you need to account for only a handful of bodies to calculate the trajectory of the satellites.

Hence, by analogy the local method in data science would be those where you deal with immediate surroundings, whatever it means in the context. For instance, kernel methods would be one example. Here, although all observations go in, they are weighted by the distance to the given observation: farther the distance, the smaller is the weight. This way, we make the interactions local. Nearest neighbors methods are obviously quite local by construction.

There are nonlocal phenomena in physics, where the disturbance would be transferred faster than the speed of light. For instance, entanglement in quantum mechanics can be interpreted as such.

Aksakal
  • 55,939
  • 5
  • 90
  • 176