1

I'm trying to differentiate two groups of patients using various machine learning algorithms, including support-vector machines (SVM).

As far as the details of the analysis go, I would like to train the sample on a separate group and cross-validate on another.

The problem is that patients are different in some categorical variables (gender for example) and continuous variables (age for example) none of which are of interest. In regression analysis using generalized linear models, it is easy to factor out nuisance variables. I'm wondering whether there is a way in machine learning as general, and SVM in particular to factor out the effect of nuisance variable. In some papers I have seen that authors include nuisance variable to somehow normalize them.

Glen_b
  • 257,508
  • 32
  • 553
  • 939
Arman
  • 161
  • 9

1 Answers1

1

In general, you can run a feature selection algorithm to preprocess the data and remove irrelevant features.

SVMs are quite robust against non-contributing features (noise). You shouldn't really care of removing features manually, it will happen "automatically". On the other hand training time will increase since since finding a solution will be harder. In the primal (linear kernel) you expect unimportant features to receive low weights (close to zero) compared to the important features; in the dual similar effect although more difficult to interpret the final model (since you don't have feature weights, but similarity to training data).

I assume you don't have enough data, thus to avoid the introduction of bias: you need to utilise a resampling method (e.g. bootstrapping) and combine it proper separation of data in training/validation/test sets. Check answers in "How to evaluate/select cross validation method?". Bootstrapping is implemented in "boot" package of R.

Community
  • 1
iliasfl
  • 2,514
  • 17
  • 30
  • For example, 2 different cohorts of patients from two different countries would have variables of interest= medical imaging data, outcome =the diagnosis. Here nuisance variables could be:age, sex, different medications. Feature selection is performed on training data. So how could it "cancel-out" the effect of nuisance variables on validation data? It may well be that patients in validation data are more clustered than training data and we get perfect performance, unrelated to variables of interest. – Arman Jan 28 '14 at 09:45
  • For example because patients in validation set are older, they are more clustered, because their age make them more different than the younger training data. I'm sure feature selection on training and validation set is a nice fishing expedition. – Arman Jan 28 '14 at 09:50
  • Also check this page http://stats.stackexchange.com/questions/81973/how-to-evaluate-select-cross-validation-method/81994#81994 about how the bias (what you refer as clustered) can be avoided. – iliasfl Jan 28 '14 at 10:13
  • I am not sure I am following, probably some terminology issues. So, whatever you call nuisance variables (age, sex etc) make perfect sense to me to be treated as normal variables and predictors. Simply because they can contribute information towards the prediction. What you say about bias in validation set means that your experimental framework is NOT strong enough. Since it's medical, I assume you don't have enough data, thus: you need to utilise a resampling method (e.g. bootstrapping) and combine it proper separation of data in training/validation/test sets. – iliasfl Jan 28 '14 at 10:14
  • Thanks a lot. Could you please add your answer below. Bootstrapping and I pick it as the best answer. If possible refer to appropriate r package. Thanks again – Arman Jan 28 '14 at 11:55