-2

Premise

It's a non-argument that linear algebra is a fundamental tool for many fields that use statistics. That being said, how fully one needs to grasp it often varies. Computers are very good at solving linear algebra and there are a few very popular programming languages that help humans harness that throughput power. Here is one of my favorite quotes on this:

If you are satisfied with merely being a consumer of scientific software, you can skate by without much knowledge of linear algebra at all.

Now I'm not arguing that humans should ever try to solve linear algebra instead of computers. I mean just that with a basic high-level conceptual understanding, one could presumably go far. Whenever we call .fit() there is a lot of linear algebra happening under the hood. The caveat here could be that IF we ever wanted write an algorithm or run a regression that was not built-in, we'd have to cowboy-up and wrangle our own linear algebra implementation. And lastly, it's worth noting, having a solid understanding of linear algebra will make it easier to troubleshoot and debug code when/if errors are thrown.

So I'm not disputing that linear algebra is useful-- whether regarding the concepts or the mechanics. What I fail to see, however, is: is linear algebra ever used for post-regression/post-convergence tasks?

Question

After all the heavy-lifting of solving the regression and/or machine learning algorithms of our data using linear algebra is done, is that always the end of the story for linear algebra? Is it just a tool for the drudgery of rows/columns or can it ever be used after a regression, let's say: for inference or related tasks?

Further Clarifications:

  • Assume we have a vector of coefficients from our regression (the final coefficients, we have already converged -- doesn't matter which model, if you are picky: ridge regression)
  • use case: inference (we don't need to use linear algebra to feed the coefficients to anything further down the pipeline; we're not landing a robot on Mars or anything) I might make an exception if you feed this vector to something else that helps the cause of inference.
  • basic linear algebra transformations for the sake of exporting data don't count. (for example in numpy we would might need to remove headers or transpose the features matrix if we wanted to save as a .csv or other file format)
Arash Howaida
  • 741
  • 7
  • 19

2 Answers2

3

Yes.

  • We will need to apply linear algebra to combine the fitted parameters $\hat{\beta}$ with the future design matrix $\hat{X}$ to create mean point forecasts, $\hat{X}\hat{\beta}$.

  • Or to calculate a predictive distribution using the predicted variance, e.g., in OLS.

  • Or to set up and apply contrasts to test one or multiple hypotheses, e.g., as in the multcomp::glht() function in R.

Stephan Kolassa
  • 95,027
  • 13
  • 197
  • 357
2

After all the heavy-lifting of solving the regression and/or machine learning algorithms of our data using linear algebra is done, is that always the end of the story for linear algebra?

No... You should always analyze your statistical results, and your interpretation could be heavily related to linear algebra.

Give you a common example. I recently analyzed a multiple linear regression problem. The results were unexpected. It was caused by instability in matrix operations. You should learn linear algreba for statistical modelling.

SmallChess
  • 6,764
  • 4
  • 27
  • 48