1

According to arima(p,0,q) model if we have n data and our total parameter is p+q then it is said that degree of freedom is n-(p+q). Could you mathematically demonstrate it? No sufficient information on internet and books.

Regards;

mertcan
  • 85
  • 1
  • 7

1 Answers1

-1

The degrees of freedom in a model is the number of obervations minus the number of parameters to be estimated. So, there is nothing to be proven because it's a definition.

mlofton
  • 1,995
  • 1
  • 9
  • 16
  • But this situation can be explained mathematically in multilinear regression model using idempotent matrix. Are you sure that there is no proof that ARIMA(p,0,q) model has n-p-q degree of freedom? – mertcan Apr 01 '19 at 10:09
  • I do not only want to memorize arıma model degree of freedom is n-p-q. Could you help me understand via mathematical proof? – mertcan Apr 01 '19 at 21:40
  • Hi:IIRC, the idempotentcy proof in multiple regression proves the unbiasedness of the estimate of sigma squared in multiple regression. But that's still defining the dof as number of samples - number of parameters estimated. In the arima case, the degrees of freedom is not used to prove unbiasedness so I'm not clear on what you're asking. – mlofton Apr 02 '19 at 14:05
  • I believe there must be analytical proof that ARIMA(P,0,Q) has n-p-q degree of freedom. Otherwise it has no meaning actually. I just would like to proofs. Could you help me? – mertcan Apr 02 '19 at 15:09
  • Hiu: Unfortunately, there is no proof, whether it be regression or time series. The imdempotency result you are referring to ( which is used in the proof ) is used to prove that the formula for the estimate of $\sigma^2$ is unbiased. Look in any decent regression or time series book. There won't be a proof because the dof is DEFINED that way. – mlofton Apr 03 '19 at 22:44
  • [in the link](https://stats.stackexchange.com/questions/24921/distribution-of-sum-of-squares-error-for-linear-regression/400261?noredirect=1#comment749409_400261) there is no proof related to unbiasedness instead using idempotent matrix sum of squared errors divided by $sigma^2$ is chi square dist. with n-p-1 degree of freedom. Actually there is a derivation of degree of freedom of chi square distribution using idempotency. What would you say about that? – mertcan Apr 04 '19 at 17:28
  • Hi: That's a proof that, assuming normaility of the error term, the SSE in a multiple regression is chi squared with n-(p+1) degrees of freedom.. it's not proving that the degrees of freedom in a regression model is n-(p+1). They ( the chi squared dof and a model's dof ) are two TOTALLY DIFFERENT THINGS. I honestly don't mean to be rude but if you take an upper level class in mathematical statistics, you will understand what that proof is used for and what the difference is. That's also where you will prove the unbiasedness of the estimate of $\sigma^2$. – mlofton Apr 05 '19 at 12:37
  • 1
    I do not think this is the definition of degrees of freedom. The notion of degrees of freedom is more complicated than that. There are models/techniques where this does not hold, e.g. k nearest neighbours (see James et al. "Introduction to Statistical Learning" and/or [nearest neighbors degrees of freedom](https://stats.stackexchange.com/questions/50379/nearest-neighbors-degrees-of-freedom)). If I remember correctly, ridge regression would be another example. @mertcan – Richard Hardy Jun 15 '19 at 06:41
  • Hi Richard: What I described is how I learned it ? I'm definitely no expert with degrees of freedom ( I always found it confusing ) so I won't comment one way or the other except to say that there may be multiple definitions. Thanks for comment. – mlofton Jun 16 '19 at 14:12
  • This showed up in review as low quality; I agree with @RichardHardy that degrees of freedom is deeper than "that's just how it's defined." There is a very readable article available as pdf by Helen Walker caleld simply "Degrees of Freedom" that goes into the reasons why degrees of freedom are as they are in linear models. – einar Jan 23 '20 at 08:48
  • I'll check it out. I took the question from the OP to mean "how do I prove that the degrees of freedom is n - (p+q) in an arma(p, 0, q)". I don't think that it's possible to prove it but still her article could be enlightening. thanks. – mlofton Jan 23 '20 at 10:13