5

I am estimating a GMM IV model, where I'm creating a weighting matrix by taking the inverse of Z'Z, where Z is a matrix of instruments. For certain combinations of instruments, when I try to compute this in R with the solve() function, I get an error along the lines of: 

system is computationally singular: reciprocal condition number = 4.96805e-21

Sometimes there's an obvious reason for the singularity (i.e., I've included something stupid in the instrument set), but sometimes there's no obvious reason for it. Now, I know I can override the error by setting the tol option to something smaller. What I want to know is: under what circumstances is this a reasonable thing to do (for instance because R's default tolerance is too high for my type of problem), and under what circumstances is this going to get me in trouble?

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
rsandler
  • 151
  • 6
  • 2
    It's best to spell out your abbreviations, like "GMM" and "IV", the first time you mention them, so it's clearer what you're referring to. For instance, I suspect in your case "IV" means "instrumental variable" but "IV" is also regularly used to refer to an "independent variable". Another advantage of doing this is that someone searching for "instrumental variable" can find your thread. (It would also help if you add a few more relevant tags - while it's true this is a problem about matrix inverses, that's not the only thing going on here.) – Silverfish Sep 16 '16 at 14:11

2 Answers2

1

Let $A$ be the matrix that R claims is computationally singular. For the purpose of this answer lets assume that it is really mathematically non-singular but close to singular (otherwise lowering the tolerance will do nothing). Let $\epsilon$ be the tolerance in use. For condition numbers see: https://en.wikipedia.org/wiki/Condition_number. The condition number is a (maximal bound on) the quotient between relative error in result and relative error in the data. That is, it is a kind of derivative! But since a matrix can be changed in many different directions, this derivative could (will) depend on in which direction you perturb the matrix, and the effect on the relative error in solution could be large. If you are lucky, the dependence for the directions that are relevant to you could be small! The condition number is then the maximum of all this directional derivatives. The condition number given in your R output is about $\kappa =.5\cdot 10^{21}$.

In R all calculations are done in double precision, so machine epsilon eps is about $10^{-16}$. Suppose you change the input data with about an eps, then the output could change with as much as $\text{eps}\cdot \kappa= .5\cdot 10^{5} $. So you could ask yourself if you are comfortable with that!

Or you could investigate it with your data, by simulation. First find a tolerance low enough that R will give a solution (if that is impossible, nothing can be done). Then, a few times add some random noise to your input data, with a variance determined by the level of measurement noise in your data. Run the analysis in R with your perturbed data, and see how much the output changes. You can use that to determine if the lowered tolerance is acceptable for you. A very few simulation runs should be enough.

For the case of linear models, there is an R package on CRAN, perturb, that helps with this.

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
0

I know this is an old question but I've recently found this to be problematic with my work.

This issue has occurred for me when trying to overfit in regression analysis. If I purposefully try to overfit a regression line, my function crashes and responds:

Error in solve.default(t(X) %*% X) :

system is computationally singular:reciprocal condition number = 2.47244e-21

If I add the constraint toll = NULL in the solve() command within my function it allows me to overfit the line and to obtain any parameters of interest.

To answer the first question, for me this was an academic exercise and could be useful if you're analysing new control algorithms in engineering, or creating your own algorithms for the purpose of machine learning.

For the second question; it's useful in some cases to modify the constraint, but I'd recommend doing so on a case by case basis

user18483
  • 31
  • 3