2

Regression is often given as a simple example for supervised learning because you have a dependent variable and try to build a model with the independent variables.

Could you say that correlation is a simple example of unsupervised learning because you don't differentiate between dependent and independent variables and try to find a pattern in all variables simultaneously (like with clustering which is always given as the example of unsupervised learning)?

vonjd
  • 5,886
  • 4
  • 47
  • 59
  • 4
    Can correlation be considered a type of learing at all? Correlation is a parameter of a bivariate distribution, just like mean or variance. Is mean a type of learning then, too? I don't know the answer, so this is an honest question. (Not that I would have an easy time with the [regression model](https://stats.stackexchange.com/questions/173660/definition-and-delimitation-of-regression-model).) – Richard Hardy May 05 '18 at 09:35
  • @RichardHardy: Another observation is that there are many similarities between linear regression and correlation - see also here: https://stats.stackexchange.com/questions/2125/whats-the-difference-between-correlation-and-simple-linear-regression and here: https://stats.stackexchange.com/questions/108640/similarities-and-differences-between-correlation-and-regression – vonjd May 05 '18 at 10:03
  • @RichardHardy: See also my answer here: https://stats.stackexchange.com/a/344619/230 – vonjd May 05 '18 at 11:08
  • 1
    @RichardHardy: I don't want to get too philosophical but in the end learning in the ML sense is always nothing else but finding some (statistical) parameters of the data. – vonjd May 05 '18 at 11:38
  • @vojd, I think it is not the case, and there is a crucial difference between statistical modelling and machine learning. A statistical model always addresses some parameters of the distribution of the data (conditional mean, conditional variance, conditional distribution, ...). Meanwhile, even though a machine learning model contains some model parameters, they need not be related to distributional parameters at all. For example, the weights of nodes in a neural network have nothing to say about the distribution of the data (or do they?). – Richard Hardy May 05 '18 at 19:06
  • @RichardHardy: To be honest with you, I think in the end they indeed do. All attempts to differentiate between statistical modeling and ML I have seen so far have an element of arbitrariness. – vonjd May 05 '18 at 19:43
  • Well, I do not quite see how. And if they do, that might just be a coincidence. (Because, yes, inevitably there are some relations between the two.) This applies to other machine learning methods as well. The goals of ML vs. statistics are fundamentally different. Statistical models are about distributions, and the technical goal is often to maximize the likelihood (in the classical paradigm). Meanwhile, machine learning is mostly about prediction, and the technical goal is minimizing loss functions which are not necessarily linked to distributions and probability densities. – Richard Hardy May 05 '18 at 19:54
  • Thus, the starting points and the goals of statistics vs. ML are different. There are similarities in the process; there may be models that fit both goals. But that might be more of a coincidence and that does not mean the goals and perspectives are alike. It think there is a difference. Perhaps this sheds some light on my very first comment, too. (And by the way, I am sort of improvizing here, trying to figure what I think about this myself. The topic does not come up too often, so I do not have a fully settled opinion.) – Richard Hardy May 05 '18 at 19:55
  • @RichardHardy: Could you give me a good reference which goes deeper into the different approaches on a meta level? I would be very interested - Thank you – vonjd May 05 '18 at 19:59
  • 1
    I don't have one, sorry about that. Perhaps there are some in earlier threads. There must be. Perhaps [here](https://stats.stackexchange.com/questions/6/the-two-cultures-statistics-vs-machine-learning). – Richard Hardy May 05 '18 at 20:01

1 Answers1

1

Yes. I would say so. Typical other example for unsupervised learning is density estimation and clustering. To me, there is on a meta level no difference between estimating a density, estimating labels to learn clusters and estimating a correlation.

Jonas
  • 716
  • 4
  • 12
  • 1
    However, I don't like the term "learning". I agree that these are "unsupervised" as in they are not given an explicit value to predict. But is the use of the tent "learning" really justified? Correlation is just a number. – Has QUIT--Anony-Mousse May 06 '18 at 17:07