2

The Minimum Covariance Determinant (MCD) estimator may be used to achieve robustness when estimating a covariance matrix. It looks for the subset of $h$ data points (out of $n > h$) whose covariance matrix has the smallest determinant and it computes the covariance of those $h$ points.

Why does minimising the determinant do the job?

cardinal
  • 24,973
  • 8
  • 94
  • 128
user7064
  • 1,685
  • 5
  • 23
  • 39
  • i think i answered this question in the paragraph under formula [2] in http://stats.stackexchange.com/questions/21907/what-is-the-difference-between-the-mcd-and-the-mve-estimators ... – user603 Jan 30 '12 at 06:33
  • Sorry @user603 but I am not totally satisfied with your comment. Let us say that the subset of h points contains an outlier. I still do not understand why the determinant will be large... thank you – user7064 Jan 30 '12 at 08:59
  • but I never claimed such a thing...if you looked at assyptotic bias curves (say in the paper I linked to in my previous answer) you would see that this is not a particularly original point either. – user603 Jan 30 '12 at 10:34
  • Sorry... definitely, I do not get your point. If you still have some time to explain me the rationale behind the MCD estimator, can you do it independently from the other question please. I really don't understand why we try to minimise the determinant to achieve robustness... – user7064 Jan 30 '12 at 11:02
  • Technically, 'we' don't try to minimize the determinant to achieve robustness but to achieve high break down point (which is a provable property of an estimator). It turns out that volume minimizing estimators (and hence also truncated likelihood ones like MCD) achieve this property. Are you asking for a proof that MCD has high break down point? – user603 Jan 30 '12 at 12:45
  • Well... yes, I would be happy to see it... is it possible to have it with respect to the MCD estimator, not with respect to the whole class of volume minimising estimators? – user7064 Jan 30 '12 at 12:57
  • ok, i'll try to outline some of the steps intuitively (or maybe someone else here?) later this week...but you should consider consulting the underlying book (Rousseeuw, P.J. and Leroy, A.M. (1987). Robust Regression and Outlier Detection. Wiley-Interscience, New York. ISBN 0-471-85233-3.) its dealt with in the (next to?) last chapter on MV-estimators (plus the material on the concept of break down point in the early ones). – user603 Jan 30 '12 at 13:00

1 Answers1

1

Okay, i tried an intuitive overview.

Suppose that $X\sim F_p(m,C)$, a known continuous distribution on $\mathbb{R}^p$ (often assumed elliptical but this can be generalized). So $X$ is a collection of $n$ draws from $F_p$.

I show it for the estimator of scatter, the proof for the estimator of location follows easily.

Denote $S(X)$ an estimator of $C$ computed from the uncontaminated data $X$, $S(X_{\epsilon})$ another estimator of $C$ computed on $X_{\epsilon}$, the contaminated data. $X_{\epsilon}$ is $X$ but for $\lfloor n\epsilon\rfloor$ of the observations which have been replaced by draws from $G_q$.

  • max-bias of an estimator $S$, for a rate of contamination $\epsilon$ of a dataset $X$:

$\verb+bias+(\epsilon,S,X)=\displaystyle\underset{X_{\epsilon}}{\sup}||S(X)-S(X_{\epsilon})||_{\kappa}$

where $||.||_{\kappa}$ is the condition number metric.

  • breakdown point of an estimator $S$:

$\epsilon^*(S,X)=\min\{\epsilon:\verb+bias+(\epsilon,S,X)=\infty\}$

The max bias is often hard to manipulate theoretically because it depends on the distribution of $X_{\epsilon}$, whereas the definition of the breakdown point contains no probability distribution.

  • maximum breakdown point of an affine equivariant estimator $S$

For an AE estimator S and a dataset $X$ in general linear position in $\mathbb{R}^p$:

$\epsilon^*(S,X)< \left\lfloor \frac{n-p-1}{2n}\right\rfloor$

  • breakdown point of an affine equivariant estimator $S$

For an AE estimator S and a dataset $X$ in general linear position in the expression of the breakdown point simplifies to

$\epsilon^*(S,X)=\min\{\epsilon:\frac{\lambda_1(S(X_{\epsilon}))}{\lambda_p(S(X_{\epsilon}))}=\infty\}$

where $\lambda_p(C)$ ($\lambda_1(C)$) is the smallest (largest) eigen-value of a covariance matrix $C$.

There is then two types of breakdown. Implosion breakdown are caused by cases where $\lambda_p(S(X_{\epsilon}))=0$ (cases where $h$ observations lie on an affine hyper-plane) and explosion breakdown by cases where $\lambda_1(S(X_{\epsilon}))=\infty$.

The exact fit property of the MCD protects it from implosion breakdowns. For explosion breakdowns, it is enough to show that the chosen MCD H-subset $H_*$

  • $H_*=\underset{H:|H|=h}{\Arg.\min.}\det(\underset{i\in H}{\text{cov}}(x_i))$

cannot belong to the set of $H$-subsets $H^+$:

  • $H^+=\cup \{H:|H|=h, \lambda_1(\underset{i\in H}{\text{cov}}(x_i))=\infty\}$
user603
  • 21,225
  • 3
  • 71
  • 135