Problem on time-series

Question

I am dealing with event data (recorded over a month) which gives out a binary response from a sensor when a door opens or closes - the time is noted at every instant and can also be represented in epochs. The times are non-uniform and the data doesn’t have any apparent pattern.

I am building an algorithm which can take the event data, and convert it into a time series. The objective is to detect the anomalies and the most frequent periodic patterns. Finally, it should also be able to predict the anomalies and their time of occurence in the near future.

The literature I have gone through has advocated the use of ARMA models and neural networks in order to handle this stochastic time-series problem. I have converted the event data into time series by measuring each time of opening and its duration.

X = Duration of the opening , t = Time of opening

However, I am not able to figure out the prediction part - should I prefer using ARMA models or any other kind of analyses like wavelets? How should I go about using neural networks? (I can solve this on MATLAB)

Any discussion on this front will be greatly appreciated. Please help.

Thanks!

Answer 1

Prediction is intimately related to the underlying generating process, which is unknown. As a researcher, on has to elaborate assumptions depending on what he knows and/or thinks it's a reasonable behavior for the process.

E.g.

• Is there any reason to think there might be linear persistence of shocks? Use AR components
• Is there any reason to think that shocks' impact totally fade off after some observations? Use MA components
• Is there any reason to think that shocks' impact is time-clustered, so that it depends on a given state of the process? Use Markov-chain switching models
• ...
• Any reason to think that there are mixed effects? Use mixed models, ARMA, ARIMA, GARCH, Markov-Switching AR, etc. Always remember, however, that the more the model is comples (and e.g. Markov-swithing complexity increases very fast with the margin increase of paramters), the more you are likely to over-fit your data.

A good literature on model families is in Forecasting Economic Times Series (Clements, Hendry), but I am sure there are tons of other very good books on this.

To get back directly to your problem, I would proceed this practical way:

1. Formulate hypothesis on the underlying process
2. Calibrate few models as identified by output of 1, analyzing residuals (most of the models assume normalit)
3. For those models seemingly fitting more the data, run pseudo-out-of-the-sample forecasting experiments for each identified model (can be done nicely with MatLab)
4. Depending on results of 3, elect the "winning model" to use in real world forecasting, defined at the one that minimize a given measure of forecasting error (e.g. MSFE)

Answer 2

I recently had to use ARIMA (simular to ARMA) and NN to do some time series prediction myself. I can't answer your question fully but I can give you some places to start. I should say I'm no expert.

First tools

There is a great NN library called FANN http://leenissen.dk/fann/wp/ its nice and mature and easy to use. Its a C / c++ library.

As for ARMA / ARIMA I used the Forecast package for R http://robjhyndman.com/software/forecast/ it also contains many other forecasting tools (but not NNs). It contains nice features which try to optimise the ARIMA models for you (which is great if you are new to the topic).

Form of data

It wants to be a series of evenly spaced (in time) values. For instance whether the door was open or closed each second.

Other issues

The fact its binary data might be interesting... But its worth a go :)

Good luck