0

$A^TA$ need not be invertible. If columns of a matrix A are linearly independent, then $A^TA$ is invertible.

Let's say $A^TA$ isn't invertible:

Gauss-Markov seems to hinge on $A^TA$ being invertible. Suppose that A is $mxn$ real matrix. If $m<n$, then the inverse of $A^TA$ does not exist.

But you can minimize the sum of squared differences, regardless (use some iterative algorithm). The proof for Gauss-Markov that I know won't work anymore. (BLUE: best linear unbiased estimator, OLS: ordinary least squares). In this situation I could see having multiple solutions. And for any of these solutions, are they the OLS? Are they BLUE? Why, if Gauss-Markov doesn't work anymore.

Gauss-Markov: https://en.wikipedia.org/wiki/Gauss%E2%80%93Markov_theorem Convexity of Linear Regression: Convexity of linear regression

eternalmothra
  • 145
  • 5
  • The last few paragraphs of this question wander off into what looks like a triviality or something without meaning: typically, $A$ has many more rows than columns, so it doesn't even make sense to refer to its "inverse." What do you really want to ask? – whuber Apr 28 '17 at 15:35
  • Thank you for the edit. Have you investigated the standard solution to least squares? You seem to be aware it can be expressed in terms of the inverse of $A^\prime A$, but since that explicitly shows there is a unique solution, it's hard to determine what you're trying to ask. – whuber Apr 28 '17 at 15:51
  • I'm trying to clarify; there maybe 2 questions here. So if $A^TA$ ISN'T invertible, can you still get the OLS solution? I think also I wasn't convinced that there was a unique solution if $A^TA$ was invertible. Does that make sense? – eternalmothra Apr 28 '17 at 16:03
  • I think the second question was too confusing, so I just removed it. I will try to restructure it and perhaps make a separate question later. – eternalmothra Apr 28 '17 at 16:13

0 Answers0