I recently heard over a radio program (in French) that a given problem - can't recall it exactly, but it involved solving large systems - could be solved "through matrix inversion" or "through an iterative procedure", with both procedures arriving at the same result.
Could anyone provide an example on where a problem could be solved by either approach to arrive at the same answer? Also, is it correct to describe the 'matrix inversion' way, the 'analytical approach'?
A common example that arises in a statistical context is linear regression. For this model the OLS estimator for the regression coefficients is given by:
$$\hat{\boldsymbol{\beta}} = (\mathbf{x}^\text{T} \mathbf{x})^{-1} (\mathbf{x}^\text{T} \mathbf{y}).$$
This formula involves the inverted Gramian matrix $(\mathbf{x}^\text{T} \mathbf{x})^{-1}$, so you can compute it using matrix inversion if you want to. However, inversion of matrices is one of those nasty things we try to avoid; it is computationally expensive (i.e., it takes a long time) and it can lead to "underflow" problems (i.e., problems arising from rounding error). For this reason, it is usual to solve linear systems like this without using matrix inversion. There are various ways this can be done, including using iterative methods for solving linear systems.
As to what is the "analytical" approach, I'm a bit skeptical of that terminology here. Even if you invert the matrix, this computation will almost certainly involve an underlying iterative procedure (unless the matrix is small) and so it is not really any more "analytical" than iterative methods that operate on the linear system instead. Also, bear in mind that different procedures will arive at slightly different results due to rounding error in the computations. The results of correct procedures should be extremely close to one another, but they might not be exactly equal.
