3

I'm trying to work out how you do this from first principles. The Wikipedia page on linear regression gives me enough to solve it with matrix operations through the origin but I can't find much literature on implementing an algorithm for an OLS fit returning coefficients, a t-statistic and an $r^2$ value for the fit

Can anyone point me to good reference?

Thanks

Chris
  • 31
  • 1
  • 1
    Is the question about understanding the mathematics and/or mathematical formulas behind the mentioned quantities or is it about *numerical algorithms* to actually *compute* them? Strangely enough, they're *very* different questions. – cardinal Nov 05 '11 at 21:43
  • Both really. I'm primarily in need of implementing it, but I would like to understand the maths properly too. – Chris Nov 05 '11 at 22:38
  • Let me ask a question so that I understand yours better: *Why* do you need to implement it yourself? There are (many!) very good reasons *not* to do so, and instead to rely on available libraries and/or software packages with many tens (or hundreds) of thousands of man-hours already put into them. – cardinal Nov 05 '11 at 22:58
  • For the mathematical and statistical background of linear regression, any good linear-regression theory text will do. [Here](http://stats.stackexchange.com/questions/17899/looking-for-mathematical-account-of-anova) is a recent question where three are mentioned. – cardinal Nov 05 '11 at 23:03
  • Well, one because I feel implementing anything is a great way to understand it properly, and two because I can't find an existing implementation for the platform I require it. – Chris Nov 06 '11 at 10:28
  • What platform are you trying to do this on? You can compile R from [source][1] if it isn't one of the [supported platforms][2]. Have you tried compiling it for whatever environment you're trying to run this on? [1]: http://cran.r-project.org/sources.html [2]: http://cran.r-project.org/doc/manuals/R-admin.html – Chris Simokat Nov 06 '11 at 22:15
  • As for the theory there are numerous text books that discuss it at various lengths. Here is a previous discussion of some that are [open source textbooks][3]. You might also find some references I pointed to in [this post][4] useful to you as well. [3]: http://stats.stackexchange.com/questions/614/open-source-statistical-textbooks [4]: http://metaoptimize.com/qa/questions/5123/what-is-the-non-probabilistic-justification-for-least-squares-regression#5125 – Chris Simokat Nov 06 '11 at 22:15

1 Answers1

2

There are quite a few books that cover this.

My personal favourite is "Solving least squares problems" by Charles L. Lawson and Richard J. Hanson.

A more recent work is "Numerical methods for least squares problems" By Åke Björck.

NPE
  • 5,351
  • 5
  • 33
  • 44