2

I am trying to predict the traffic volume for all the stations. My dataset consists of two response variables (double), the total traffic volume and the net flow traffic volume. There are 5 explanatory variables: the station id (factor), year (factor), month (factor), day (factor), hour (integer), weekday (binary). Due to the fact that there's no obvious linear increment of the traffic volume against year/ month/ day, these variables are treated as categorical ones.

I would like to use the 6 input variables to predict for the total traffic volume for each station and the net flow separately. Simple linear regression could never finish the computation. Random forest could not handle categorical predictors with more than 53 categories.

Could anyone help and point me the right direction about what models might be able to solve this problem in R?

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
v15
  • 21
  • 2
  • 6
    Time variables should not be a factor variable. You need some kind of time series analysis which can take into account seasonality. – Roland Apr 10 '17 at 14:12
  • Just as Roland commented, month, day, hour variables especially seem to have seasonality on traffic y-variable where if you put them into regression model, residuals are highly auto-correlated that the result will be poor. You might have to plot them to check their moving patterns and apply time-series analysis. –  Apr 10 '17 at 14:28
  • You could look into fused lasso – kjetil b halvorsen Apr 10 '17 at 15:59
  • Just how many categories are there? – whuber Apr 10 '17 at 16:46
  • There are around 600 stations, 3 years, 12 months and 365 days. – v15 Apr 10 '17 at 18:37
  • 3
    That gives you around 8000 observations per station--that ought to be enough for many purposes! Why not run a separate model for each station? – whuber Apr 10 '17 at 19:33
  • https://stats.stackexchange.com/questions/227125/preprocess-categorical-variables-with-many-values/277302#277302 – kjetil b halvorsen May 03 '17 at 14:04
  • 2
    First I'd try to model the traffic for a *single station only* with a time-series approach. – knb Sep 05 '17 at 10:44

2 Answers2

1

There are so many models can handle this kind of problems. Decision Tree algorithm (it can also include regression analysis) can be useful. Neural Network is always powerful but also computationally expensive. Logistic Regression is another approach, that I personally would follow as the first step since it is relatively powerful for prediction problems and does not consume so much computational resources relatively compared to aforementioned techniques.

Just to have a list of available algorithms in R (that I have used):

Random-Forest

Logistic Regression

Naive Bayes

Neural Network

There are some other methods available that I am not familiar with therefore won't recommend. Another suggestion is using some measures like Coefficient of determination before converting date (i.e. year, month, day) to categorical variables (@Ronald's comment is helpful).

Good luck!

M--
  • 115
  • 6
1

If you run a separate model for each station as whuber suggests in comments and bearing in mind that as roland suggested in comments you probably want to fit a model which takes account of the fact that your predictor variables are times and not just standard covariates then you will end up with 600 sets of coefficients and their variance covariance matrix. You can then combine these using standard multivariate meta-analysis methods using the estimates and their variance covariance matrix. Since you use R you might investigate some of the examples using metafor here

mdewey
  • 16,541
  • 22
  • 30
  • 57