4

I am currently working on a topic where I know that the distributions of the output and of the covariates will shift. I know for example that some covariates will at least follow the inflation rate. The goal is to predict the output over time, based on time series. As far as I understand machine learning methods, they suppose constant distributions over time.

I thought about adapting the standardization of the covariates over time but I am not sure it would work.

Is there a method used in this case to adapt the model (regression trees or SVR for example) to take that into account ?

I think this issue can be divided in 2 subparts:

1. Is there a way to adapt the model knowing approximately by how much the distribution should shift?

2. Is there a way if we do not know the potential shift?

DaL
  • 4,462
  • 3
  • 16
  • 27
LouisBBBB
  • 193
  • 13
  • Can your problem be recast as a problem of extrapolating from a nonparametric model? For example, say you have $y = f(time)$. If you fit this with OLS, you can extrapolate. If you fit it with a kernel regression (or a random forest or any other NP algo), you can't extrapolate all that well. Is your problem fundamentally different from this? – generic_user Feb 17 '17 at 10:09

1 Answers1

1

What you are referring to is concept drift which has a lot of research being done in the context mostly of online learning.

To familiarize yourself with the research I would start at this review paper (13 years old now) and see related articles.

Generally speaking it's a very hard to learn under concept drift because the iid assumptions break, the challenge is to detect when that happens.

Bar
  • 2,492
  • 3
  • 19
  • 31
  • 1
    See also [A Survey on Concept Drift Adaptation][1] and [Learning under Concept Drift: an Overview][2] as overviews. [1]: http://www.win.tue.nl/~mpechen/publications/pubs/Gama_ACMCS_AdaptationCD_accepted.pdf [2]: https://arxiv.org/pdf/1010.4784.pdf – DaL Apr 18 '17 at 11:48