5

is there any theoretical result which says that use the minimum of the cross-validation as value for the lasso penalty is a good choice?

I would like something like $P(S_0 \subset \hat S_{lasso}(\lambda_{cv}))\rightarrow 1$ where $S_0$ is the set of true variable.

Where can I find it?

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
Donbeo
  • 3,001
  • 5
  • 31
  • 48
  • 1
    This is a pretty generic question about generalization error and empirical risk minimization. – hearse May 08 '13 at 16:42
  • 3
    In the book statistics for high dimensional data they said :" The empirical fact that often $S_0\subset \hat S$ is supported by theory. " Where can I found that theory? – Donbeo May 08 '13 at 16:51

1 Answers1

2

The property that you're looking for is sometimes called the "oracle property": Can we estimate the true subset $S_0$ of variables with increasing number of observations $n$?

It has been shown that the classical lasso has the oracle property only under some specific conditions (see here).

My best guess is that these conditions transfer to the case of the lasso with cross-validation.

Edgar
  • 1,391
  • 2
  • 7
  • 25