2

Let's say I'm using the Sonar data and I'd like to make a hold-out validation in R. I partitioned the data using the createFolds from caret package as folds <- createFolds(mydata$Class, k=5).

I would like then to use exactly the fold mydata[i] as test data and train a classifier using mydata[-i] as train data.

My first thought was to use the train function, but I couldn't find any support for hold-out validation. I mean, I don't want to partition my data and then test all the combinations for estimating the mean accuracy, I want to train it only once. Am I missing something here?

Also, I'd like to be able to use exactly the pre-defined folds as parameter, instead of letting the function partition the data. Does anyone have any thoughts?

Thanks in advance

gcolucci
  • 133
  • 1
  • 4

1 Answers1

3

It is hard to tell exactly what you are interested in. If it is a single hold-out, you can use 

trainControl(method = "LGOCV", p = .8, number = 1)

and 80% will be used for training.

There is also method = "none" that will just fit the model for a single tuning parameter value (using the entire training set).

Also, if you want to use your own hold-out set(s), see the index argument of trainControl.

Max

topepo
  • 5,820
  • 1
  • 19
  • 24
  • It worked!! Thanks a lot, @topepo, and great job with the package! I noted, however, that if i pass an index with the method="knn", it gives an error. Is that expected? – gcolucci Apr 10 '14 at 23:31