0

Should one be able to find out the degrees of freedom of a given statistic?

I liked this answer regarding degrees of freedom and I tried to apply it. Some examples:

  1. $(Z = )\frac{\bar X-\mu}{\sigma/\sqrt{n}} = 10$. As $n$ is fixed, $\mu,\sigma,n$ are constants. It would finally turn into $X_1+...+X_n=constant$ and if we fixed $X_1,...,X_{n-1}$, then $X_n$ is fully determined and not free to vary. So, $df=n-1$.

  2. $(T=)\frac{\bar X-\mu}{s/\sqrt{n}} = 0$. As $n$ is fixed, $\mu, n$ are constants. Let's say we fix $X_1,...,X_{n-1}$ such that $X_1+...+X_{n-1}=30$, and let's say $n=10, \mu=1$. I turn out that again $X_n$ is fully determined (Consider $a=X_n$: calculus. So: $X_n=-20$). So, again $df=n-1$.

First question: Am I right?

Second question: Isn't all the time the same? I mean when we have only one statistic the number of degrees may be at most decreased by 1, no? How come in t-test for mean difference $df=n_1+n_2-2$? Is it possible to have just one equation and think of decreasing the freedom by $2$?

Community
  • 1
user_anon
  • 827
  • 10
  • 20
  • "Degrees of freedom" has several meanings. The one most relevant in statistics (as a parameter telling us what the distribution of a statistic might be) is not an inherent property of a statistic: it also depends on a probability model. Therefore you cannot be generally right. See http://stats.stackexchange.com/questions/1692 for extensive materials on this subject. (The answer you reference, as well as many of the other answers in that thread, describes a different meaning of DF.) – whuber Aug 24 '16 at 15:43
  • How is the referenced answer different from above calculus? – user_anon Aug 24 '16 at 15:52
  • It's not. The point is that your calculations do not determine the degrees of freedom needed by correct statistical procedures; they only reflect *dimensions of mathematical constraints.* The two things sometimes agree but frequently do not--such as for procedures that have non-integral degrees of freedom. – whuber Aug 24 '16 at 15:53

1 Answers1

1

The degrees of freedom (dof) is the number of data points minus the number of equality constraints. Typically the latter is equal to the number of parameters estimated. For the sample mean, the sum of deviations from the mean must be zero, so that removes one dof. For a difference of two means, you can think of your second example as $(n_1-1)+(n_2-1)$. For regression, if you estimate $m$ parameters by maximum likelihood (or some other continuous optimization), then you have $m$ equality constraints: The gradient of the likelihood function with respect to each parameter must be zero. So there are $N-m$ degrees of freedom. (For example the sample mean is the least-squares solution of $x=\text{constant}+\text{noise}$.)

GeoMatt22
  • 11,997
  • 2
  • 34
  • 64
  • Several answers in the [referenced thread](http://stats.stackexchange.com/questions/1692) explicitly contradict this one. – whuber Aug 24 '16 at 15:54
  • 'Typically the latter is equal to the number of parameters estimated.'. Is there a reason why? In my first example (Z), there was no estimated parameter and $df=n-1$. Was I wrong? – user_anon Aug 24 '16 at 15:55
  • @whuber I do not have time right now to check your link, but will later. If my answer is wrong I will be happy to delete it, deferring to your expertise. – GeoMatt22 Aug 24 '16 at 16:05
  • It's not wrong--it just has a more limited scope of applicability than you suggest. – whuber Aug 24 '16 at 16:06