Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [degrees-of-freedom]

The term "degrees of freedom" is used to describe the number of values in the final calculation of a statistic that are free to vary. Use also for "effective degrees of freedom".

Reference: Glossary of Statistical Terms

"Statisticians use the terms "degrees of freedom" to describe the number of values in the final calculation of a statistic that are free to vary. Consider, for example the statistic s².

To calculate the s² of a random sample, we must first calculate the mean of that sample and then compute the sum of the several squared deviations from that mean. While there will be n such squared deviations only (n - 1) of them are, in fact, free to assume any value whatsoever. This is because the final squared deviation from the mean must include the one value of X such that the sum of all the Xs divided by n will equal the obtained mean of the sample. All of the other (n - 1) squared deviations from the mean can, theoretically, have any values whatsoever. For these reasons, the statistic s² is said to have only (n - 1) degrees of freedom."

$s^2_X = \frac{\sum(X-\bar{X})^2}{(n-1)}$

See also the Cross Validated thread: How to understand degrees of freedom? For "effective degrees of freedom" see Wikipedia

461 questions
315
votes
13 answers

How to understand degrees of freedom?

From Wikipedia, there are three interpretations of the degrees of freedom of a statistic: In statistics, the number of degrees of freedom is the number of values in the final calculation of a statistic that are free to vary. Estimates of…
Tim
  • 1
  • 29
  • 102
  • 189
34
votes
2 answers

Degrees of freedom of $\chi^2$ in Hosmer-Lemeshow test

The test statistic for the Hosmer-Lemeshow test (HLT) for goodness of fit (GOF) of a logistic regression model is defined as follows: The sample is then split into $d=10$ deciles, $D_1, D_2, \dots , D_{d}$, per decile one computes the following…
user83346
34
votes
2 answers

Satterthwaite vs. Kenward-Roger approximations for the degrees of freedom in mixed models

The lmerTest package provides an anova() function for linear mixed models with optionally Satterthwaite's (default) or Kenward-Roger's approximation of the degrees of freedom (df). What is the difference between these two approaches? When to choose…
doko
  • 441
  • 1
  • 4
  • 4
32
votes
1 answer

Can degrees of freedom be a non-integer number?

When I use GAM, it gives me residual DF is $26.6$ (last line in the code). What does that mean? Going beyond GAM example, In general, can the number of degrees of freedom be a non-integer number? > library(gam) >…
Haitao Du
  • 32,885
  • 17
  • 118
  • 213
28
votes
0 answers

What are "degrees of freedom"?

Possible Duplicate: How to understand degrees of freedom? I was at a talk a few months back where the speaker used the term 'degrees of freedom'. She briefly said something along the lines of it meaning the number of values used to form a…
Christine Forrester
  • 381
  • 2
  • 3
24
votes
2 answers

How should mixed effects models be compared and or validated?

How are (linear) mixed effects models normally compared against each other? I know likelihood ratio tests can be used, but this doesn't work if one model is not a 'subset' of the other correct? Is the estimation of the models df always…
dcl
  • 2,610
  • 3
  • 19
  • 30
23
votes
2 answers

What is the distribution of the difference of two-t-distributions

... and why ? Assuming $X_1$,$X_2$ are independent random-variables with mean $\mu_1,\mu_2$ and variance $\sigma^2_1,\sigma^2_2$ respectively. My basic statistics book tells me that the distribution of the $X_1-X_2$ has the following properties:…
mlwida
  • 9,922
  • 2
  • 45
  • 74
22
votes
2 answers

The definition of natural cubic splines for regression

I am learning about splines from the book "The Elements of Statistical Learning Data Mining, Inference, and Prediction" by Hastie et al. I found on page 145 that Natural cubic splines are linear beyond the boundary knots. There are $K$ knots,…
Durin
  • 964
  • 2
  • 7
  • 17
20
votes
4 answers

What does "degree of freedom" mean in neural networks?

In Bishop's book "Pattern Classification and Machine Learning", it describes a technique for regularization in the context of neural networks. However, I don't understand a paragraph describing that during the training process, the number of degrees…
Robert Smith
  • 3,191
  • 3
  • 30
  • 46
17
votes
2 answers

Explanation for non-integer degrees of freedom in t test with unequal variances

The SPSS t-Test procedure reports 2 analyses when comparing 2 independent means, one analysis with equal variances assumed and one with equal variances not assumed. The degrees of freedom (df) when equal variances are assumed are always integer…
Joel W.
  • 3,096
  • 3
  • 31
  • 45
14
votes
1 answer

Multiple linear regression degrees of freedom

The degrees of freedom in a multiple regression equals $N-k-1$, where $k$ is the number of variables. Does $k$ include the response variable (i.e., $Y$)? For example, in the model $Y = B_0 + B_1X_1 + B_2X_2$, then does $k = 3$ (i.e., 1 df each…
dorothy
  • 585
  • 3
  • 7
  • 16
14
votes
1 answer

AIC of ridge regression: degrees of freedom vs. number of parameters

I want to calculate the AICc of a ridge regression model. The problem is the number of parameters. For linear regression, most people suggest that the number of parameters equals the number of estimated coefficients plus sigma (the variance of…
Funkwecker
  • 2,432
  • 5
  • 24
  • 43
14
votes
1 answer

Distribution of sum of squares error for linear regression?

I know that distribution of sample variance $$ \sum\frac{(X_i-\bar{X})^2}{\sigma^2}\sim \chi^2_{(n-1)} $$ $$ \sum\frac{(X_i-\bar{X})^2}{n-1}\sim \frac{\sigma^2}{n-1}\chi^2_{(n-1)} $$ It's from the fact that $(X-\bar{X})^2$ can be expressed in…
KH Kim
  • 1,121
  • 2
  • 10
  • 25
14
votes
1 answer

Intuition for the degrees of freedom of the LASSO

Zou et al. "On the "degrees of freedom" of the lasso" (2007) show that the number of nonzero coefficients is an unbiased and consistent estimate for the degrees of freedom of the lasso. It seems a little counterintuitive to me. Suppose we have a…
Richard Hardy
  • 54,375
  • 10
  • 95
  • 219
14
votes
2 answers

Reporting degrees of freedom for Welch t-test

The Welch t-test for unequal variances (also known as Welch–Satterthwaite or Welch-Aspin) generally has a non-integer degrees of freedom. How should these degrees of freedom be quoted when reporting the results of the test? "It is conventional to…
Silverfish
  • 20,678
  • 23
  • 92
  • 180
1
2 3
…
30 31