2

I have a dataset for a lot of subjects (current testing dataset around 3000 subject, actual number is a lot bigger >40000).

Each subject has 13 variables. The data was measured once per year for 11-13 years (depends on the subject).

What machine learning technique should I use, to be able to forecast these variables for subjects, that have fewer years of observations (or are younger and we want to predict). So lets say, I have 12 years of observations and would like to model these variables for the 13th year, similar for 11 observations, I would like to model for 12th and 13th year.

The years are consecutive.

The data consists of simple physical traits, as in weight, age, sex(which appears in every row of data), height and fitness scores (time of 60m sprint, 600m sprint, etc).

I was thinking of somehow putting these variables into a neural net and train for every combination (11years of data, train for 12th and 13th separately, etc). But this is somewhat crude, time consuming and bad in general.

What is a better solution to this problem, using machine learning?

Richard Hardy
  • 54,375
  • 10
  • 95
  • 219
Ravonrip
  • 173
  • 5
  • See https://stats.stackexchange.com/questions/135061/best-method-for-short-time-series – Tim Sep 27 '17 at 10:01
  • Thank you. Will check the methods mentioned in that answer, but it worries me, that most methods are linear, for which I do not know, how will they work on my data. But will try them. The simple mean/median methods are out of the question as my data is either increasing, either decreasing (at least most of it). – Ravonrip Sep 27 '17 at 10:09
  • So you can consider things like [random walk with drift](https://www.otexts.org/fpp/2/3/). In general, neural networks are not the best choice for time-series data. – Tim Sep 27 '17 at 12:04
  • I would first look into standard techniques like mixed effects models with some AR(1) error structure per subject. – Michael M Sep 27 '17 at 13:39
  • @Ravonrip time series forecasting/clustering has benefited from deep learning advances in seq2seq problems from the area of NLP. Your data sounds like what economists would call 'panel' data. The following two links may be useful: https://arxiv.org/abs/1708.00185 https://datascience.stackexchange.com/questions/16174/recurrent-neural-network-on-panel-data – Mike Oct 31 '19 at 20:09

0 Answers0