3

I've just begun a machine learning course and I've been confused over my professors use of the degrees of freedom terminology. I've picked up that while in statistics talking about degrees of freedom typically assumes you are referencing the degrees of freedom of the error term (observations not taken up by your estimated parameters), machine learning uses degrees of freedom in almost the opposite sense (DoF being the number of parameters you have).

It bothers me a bit, because although I see post after post claiming machine learning is NOT statistics (in fact when asked about this, my professor's default response was "I am not a statistician"), I still fail to see how machine learning could exist without statistics, and so it surprises me that one would completely disregard the terminology (as far as almost choosing an opposite definition), of an more established discipline.

Can anyone provide some insight into this lexicographic quirk?

Thanks!

Benjamin Parsons
  • 83
  • 6
  • 1
    Doesn't it just depend on one's perspective? That is, DoF typically is the dimension of a space and when there are several spaces around (such as the space of data and parameter space in a statistical problem) one has to choose the space that's relevant for a particular concept. In fact, one can discern two distinct DoF concepts in many statistical problems: it is a bit of an accident that their values often coincide. See https://stats.stackexchange.com/a/17148/919. – whuber Dec 01 '20 at 22:52
  • @whuber Thanks for the response and the link is very helpful. Perhaps my understanding isn't nuanced enough, or is focusing too much on the usage of the word "freedom". Parameter estimation is its own dimension of space, but I do not see those estimators as being free to vary (rather determined by the data) . I admit, this is perhaps only a semantical problem, not a statistical one. – Benjamin Parsons Dec 03 '20 at 00:11

0 Answers0