34

Coming from the field of computer vision, I've often used the RANSAC (Random Sample Consensus) method for fitting models to data with lots of outliers.

However, I've never seen it used by statisticians, and I've always been under the impression that it wasn't considered a "statistically-sound" method. Why is that so? It is random in nature, which makes it harder to analyze, but so are bootstrapping methods.

Or is simply a case of academic silos not talking to one another?

Bossykena
  • 667
  • 6
  • 11
  • 1
    I wonder one thing about computer vision methods vs. statistics methods: performance in the first is a must. Maybe there's a trade-off between performance and "correctness", and computer vision and statistics have different weights for those variables. – Lucas Reis Jul 31 '12 at 15:26

4 Answers4

16

I think that the key here is the discarding of a large portion of the data in RANSAC.

In most statistical applications, some distributions may have heavy tails, and therefore small sample numbers may skew statistical estimation. Robust estimators solve this by weighing the data differently. RANSAC on the other hand makes no attempt to accommodate the outliers, it's built for cases where the data points genuinely don't belong, not just distributed non-normaly.

nbubis
  • 451
  • 3
  • 16
  • 1
    Great answer. I have seen RANSAC most used in CV to estimate homographies. This is most widely used when we know that some of the corresponding measurements are hugely unreliable. Also, real time performance and other considerations has made this technique quite popular as it can be easily parallelised. – Luca Jan 13 '15 at 11:36
8

For us, it is just one example of a robust regression -- I believe it is used by statisticians also, but maybe not so wide because it has some better known alternatives.

  • 1
    Can you give examples of alternatives? I'd like to look into that. – Bossykena Jul 21 '10 at 20:56
  • 6
    The mostly known and the simplest is the Median-Median regression, well known from smart calculators (Sigh!). Consult also Wikipedia http://en.wikipedia.org/wiki/Robust_regression and maybe CRAN's Robust task view http://cran.r-project.org/web/views/Robust.html –  Jul 21 '10 at 21:20
  • Are there alternatives to RANSAC which not only give you the unbiased regression but also the data points from which the model has been estimated? Thanks – Valerio Feb 03 '14 at 15:06
1

You throw away data with RANSAC, potentially without justifying it, but based on increasing the fit of the model. Throwing away data for increased fit is usually shun as you may loose important data. Removal of outliers without justification is always problematic.

It is ofcourse possible to justify it. E.g. if you known the data should follow a given pattern, but that there also are deviation in the data from the pattern due to error in the measurements.

NegativeFeedbackLoop
  • 46
  • 4
1

This sounds a lot like bagging which is a frequently used technique.

Zach
  • 22,308
  • 18
  • 114
  • 158
  • 4
    RANSAC is very different - in bagging, all samples are taken into account in some way. RANSAC is used in cases where up to 50% of the data should be completely discard. – nbubis Aug 27 '14 at 01:25