$(X^\prime X)^{-1}$ as a measure of estimator precision

Question

According to this discussion, the highest variance $X^\prime X$ matrix should correspond with the lowest variance $\beta$ which makes sense to me. But when I ran the following lines I got some curious results...

Y <- matrix( c(303.3,
               467.1,
               422.8,
               391.6,
               403.8,
               373.8,
               263.5,
               226.8,
               183.9,
               208.4,
               208.2,
               168.8,
               245.4,
               160.0,
               173.2), nrow=15, ncol=1, byrow=TRUE)
X <- matrix( c(1, 3, 1, 8, 1, 1,
               1, 6, 3, 3, 1, 2,
               1, 4, 1, 4, 1, 5,
               1, 5, 1, 3, 1, 1,
               1, 7, 1, 3, 3, 1,
               1, 6, 1, 3, 5, 2,
               1, 6, 1, 8, 5, 3,
               1, 5, 1, 8, 3, 2,
               1, 1, 3, 4, 2, 1,
               1, 3, 5, 2, 2, 4,
               1, 2, 3, 3, 1, 3,
               1, 1, 4, 3, 2, 3,
               1, 6, 5, 3, 1, 5,
               1, 2, 1, 5, 2, 2,
               1, 5, 1, 4, 2, 1), nrow=15, ncol=6, byrow=TRUE)

apply(t(X)%*%X, MARGIN=2, FUN=sd)

apply(solve(t(X)%*%X)), MARGIN=2, FUN=sd)

What I found curious is that $X^\prime X$ yields column standard deviations as follows

19.12503 94.55721 34.12575 100.84328 47.27120 43.32551

and the standard deviatoin of the columns for $(X^\prime X)^{-1}$ are

0.75477714 0.04729509 0.09832687 0.06428046 0.03173388 0.02858314

The former would suggest that the 4th column ($\beta_3$) would have the greatest precision but the latter would suggest that ($\beta_5$) is the most precise.

Did I goof somewhere? Which one should I trust more?

I am unable to see any connection between the operations you are performing and the thread you link to. Could you clarify what you mean by "the highest variance $X^\prime X$ matrix"? There's only one such matrix in evidence. — whuber, Mar 24 '20 at 20:57
I figured out what I was was misunderstanding. $(X^\prime X)^{-1}$ looks a lot like the covariance matrix for the betas. So my question above highlights my misunderstanding. I'm comfortable with the answer now. — financial_physician, Mar 24 '20 at 21:17

score 1 · Accepted Answer · answered Mar 24 '20 at 21:20

Figured out the answer to my question. Taking the variances along the columns doesn't make sense, the matrix itself resembles the covariance matrix and that covariance matrix is what I needed in order to understand the problem I was trying to solve. The beta that corresponding to the smallest diagonal along the covariance matrix is the beta that is most precisely estimated.

$(X^\prime X)^{-1}$ as a measure of estimator precision

1 Answers1