3

I'm not sure if this is true, but is something 'lost' when we attempt to estimate the grand mean when doing an ANOVA or a coefficient in linear regression? I've heard phrases like, you "burn up/lose" a degree of a freedom for each coefficient you want to estimate and I was wondering what are the implications of this? Are more degrees of freedom preferable for reason? I don't have a specific example, just wondering about what happens the more we try to estimate things about our model.

Nate
  • 595
  • 1
  • 11
  • 2
    "Degrees of freedom" is a difficult term because [it is a polysemic term](https://stats.stackexchange.com/questions/16921/how-to-understand-degrees-of-freedom) and [it isn't always clear how it applies](https://stats.stackexchange.com/questions/554428/where-is-our-consideration-of-nuisance-degrees-of-freedom-in-modelling). – DifferentialPleiometry Dec 14 '21 at 21:24
  • Would it help if we just stuck to an example like estimating coefficients of a linear regression model? Like I wanted to estimate 2 coefficients and I had 5 df to start with. What does loosing 2 df mean? – Nate Dec 14 '21 at 21:29
  • 2
    The issue isn't really one of "estimating more things." After all, both in (parametric) ANOVA and general linear modeling, the estimates *completely describe the full distribution of the data generation process.* What you are contemplating here is *modifying the model.* The issues that arise are discussed extensively here on CV under "overfitting" and "model identification." – whuber Dec 14 '21 at 21:29
  • 3
    @Nate there's a sense in which you can be using something up; if you estimate "null" parameters you're reducing the degrees of freedom in the error term -- the number of independent pieces of information that you can use to estimate $\sigma^2$, and the variability (and skewness) of the sampling distribution of that estimate increases. If you use all the d.f. estimating mean-effects, your estimate of it becomes $0/0$. – Glen_b Dec 15 '21 at 00:04

0 Answers0