2

I have what I believe to be sequential data. I have (dated) visits to a hairsalon and my aim is to predict show or no show in the future. I want to use cross validation but I believe that when I would use the normal cross validation strategy where I would take folds of my training data like this:

enter image description here

That I would have information leakage between my training and test sets, I believe that this strategy would not be good for all models in the above picture (except the one from the first iteration). In the second iteration for example you see that part of the training fold is in the "future" whilst part of the test fold is in the "past".

The other strategy that I consider is the one proposed here : Using k-fold cross-validation for time-series model selection. Is this necessary for my problem or not?

gunes
  • 49,700
  • 3
  • 39
  • 75
Tibo Geysen
  • 179
  • 10

1 Answers1

4

Yes, it is necessary because your data has temporal relationships. For example, let's say in folds 9-10, the trend changes, fold 10 is in your training folds and your validation fold is the 9-th fold. You'll learn some information about the trend change in the 10-th fold and use that information in your validation set, i.e. 9-th fold, in the prediction phase. This may result in optimistic/biased evaluation of your models.

gunes
  • 49,700
  • 3
  • 39
  • 75