3

Recently I'm working on some online learning algorithm (using RBF neural network ) for classification. As I read papers in this area I found there is an issue in online-learning called concept drift problem which my algorithm has and I have to find a way and solve this problem. So I am trying to find out what is the best way to tackle the issue of concept drift and I found approaches like sliding window, drift detection, ensemble learning and etc. Between these methods, ensemble learning is interesting for me so I watched this tutorial, I found out that this method is sampling from dataset and then gives it to different learning algorithms.

This method presented to solve concept drifting problem in online-learning( data coming one by one and train model then discarded) and it samples from the dataset, giving to different learning algorithm so we need to have a dataset already. Having dataset before and online-learning have conflict with each other. So is ensemble algorithm use in bach or online learning ?

deemel
  • 2,402
  • 4
  • 20
  • 37
mkafiyan
  • 237
  • 1
  • 8
  • You might consider adjusting the name of the question, since the title is quite broad and your actual question is somewhat narrow. – deemel Jul 24 '18 at 10:43
  • @Rickyfox yes you are right. I update it but in my opinion still doesn't good :D – mkafiyan Jul 24 '18 at 12:19

1 Answers1

1

Concept drift adaption is done in different ways, you listed some of them already. Generally it depends on the velocity and volume of the data stream in what manner you process it. True online learning, i.e. one-by-one processing of the arriving instances can be viable in low velocity streams, while for faster streams it can become impractical or unfeasible.
A good portion of the literature I've read on the topic use batch processing, since it is a helpful simplification.

[...] so we need to have a dataset already. having dataset before and online-learning have conflict with each other

Overall you have to differentiate between the initial training of a model and it's adaption over time. The extent and importance of these two strongly depend on your approach and application. It's not uncommon for data stream applications having a 'warm start', meaning that a portion of accumulated data is already present at the start. So the two are not conflicting at all.

Furthermore ensembles are a type of classifier that has many different implementations. At ECML last fall I heard a very good talk by Bartosz Krawczyk on the topic and I can recommend you his survey article on the matter

Krawczyk, B., Minku, L. L., Gama, J., Stefanowski, J., & Woźniak, M. (2017). Ensemble learning for data stream analysis: A survey. Information Fusion, 37, 132-156.

As well as some of his more recent publications on the topic. In the references you might find some other relevant articles.

so is ensemble algorithm use in bach or online learning ?

This depends entirely on the approach used, as is I've seen ensembles employed in both fashions.

deemel
  • 2,402
  • 4
  • 20
  • 37