1

Following the discussion :

Here's [a link] How are the standard errors of coefficients calculated in a regression?

I've got a question . Why variance of theta (regression coefficients) in theory is calculated by multiplying sigma^2 and (XT * X) ^ -1. Where sigma^2 is variance of the residuals : SSE / n-1 ,(where n- number of samples in the training set). However, in R it is computed as MSE = SSE / (n-p-1) (where p number of features) multiplied by the same matrix (XT * X) ^ -1. What is the reason? Many thanks in advance.

Michael2016
  • 11
  • 1
  • Did you mean to type beta instead of theta? – mdewey Oct 13 '17 at 13:00
  • Where did you read that "sigma^2 is variance of the residuals : SSE / n-1"? That's not what the linked thread asserts. – whuber Oct 13 '17 at 19:44
  • Yeah sorry theta = beta or in other words regression coefficients @mdewey. – Michael2016 Oct 18 '17 at 11:18
  • Yep @whuber when you derive maximum likelihood assuming that noise epsilon is normally distributed with variance sigma^2 and mean zero, I think that it means that residuals are distributed in that fashion. Am I wrong? since noise is itself residual – Michael2016 Oct 18 '17 at 11:21
  • It's not a maximum likelihood estimate: it's the least squares estimate. Ordinary Least Squares does not necessarily assume Normal distribution of the errors. – whuber Oct 18 '17 at 13:35

0 Answers0