4

After doing some reading on CrossValidated, I understood that we can use "imputation" techniques to fill in the gaps (if they are random). But I am not clear on following questions:

  1. How many consecutive gaps may make data set invalid for forecasting?

  2. How many total gaps in data set makes it as invalid.

For example I have hourly data for week, which means 188 total points in my data set

  1. Case 1: assume if we are missing 3 consecutive days of data is missing, can we still consider that data set as valid data set?
  2. Case 2: assume overall 80 data points are missing out of 188 points, can we still consider data set as valid?

I am using HoltWinters implementation in java for forecasting.

Any help would be appreciated.

kosa
  • 493
  • 3
  • 10
  • 18

3 Answers3

5

The Kalman filter is one alternative to fill in missing observations in time series. See this post as an example. The Kalman filter is a common algorithm that will be available in most languages and statistical software. Contrary to the Holt-Winters filter you have to specify a model for the data.

"How many consecutive gaps may make data set invalid for forecasting? How many total gaps in data set makes it as invalid."

I don't know a rule to measure this. I would say it depends on how much we know about the data and their context. Forecasting and, in general, the analysis of data involve a combination of our knowledge or theories and statistical methods to test our theories or find some further facts that we may have overlooked.

The amount of data or the presence of gaps may or may not be critical. For example, I have not looked at historical data about temperatures recorded in my town but I would be quite confident to give you a relatively narrow interval about the temperatures that will be observed in the next days. On the other hand, I have a data base with thousands of flight prices and at this moment I wouldn't dare to tell you whether you should buy a ticket today or wait until tomorrow.

So there is a combination of knowledge and data. On one side, we may know a lot about the data but we lack a minimal amount of data. On the other side, we may have a huge amount of data but they don't have much meaning to us. In the former case, we may decide to throw the data away and trust our expert knowledge to foresee the future. In the latter case, we may throw the data into a brute force algorithm (some kind of machine learning algorithm) and let it find patterns and forecasts for us.

Usually we are at some point in-between these extreme cases. You are the one who knows how much the available data can contribute to your knowledge and how much uncertainty will be in the forecasts.

Community
  • 1
javlacalle
  • 11,184
  • 27
  • 53
4

I am not sure what you mean by "a valid data set". Are you sure what you mean by it? There are reasons why, in a single or in multiple time series consecutive missingness would be irrelevant to the validity of an analysis, and reasons why it would be lethal to valid inference.

However, Honaker and King are at the head of practical multiple imputation within a time-series context:

Honaker, J. and King, G. (2010). What to do about missing values in time-series cross-section data. American Journal of Political Science, 54(2):561–581. (See also, the related R package Amelia II on CRAN)

It is not clear how familiar you are with multiple imputation, but it has two aims (1) to support inference that is unbiased by MAR and MCAR (i.e. to impute a set of reasonable values), and (2) in doing so to incorporate the additional uncertainty in one's analysis that is due to the presence of missing data (i.e. to incorporate the extra variation resulting from imputed values not all agreeing with one another).

Alexis
  • 26,219
  • 5
  • 78
  • 131
  • Thanks for the answer! "a valid data set", I mean should we use that data set at all to run forecasting? because it has lot of gaps. My understanding is because of these gaps, forecasting calculation will results in deviated numbers. Am I correct? – kosa Jul 09 '14 at 17:43
  • What is a "deviated number?" – Alexis Jul 09 '14 at 17:45
  • For example, if we use simple mean approach to fill in the gap (I know not ideal approach), forecast for next 24 hours will be let us assume 8 (1st hour of forecast),5 (2nd hour of forecast),9,7,8,5,6,..... this output is due to the values we filled in. But if we observe real value at 1st hour, I am positive it will be lot different. This is what I mean deviation. The difference between forecast value and real observed value at a given future point. Am I thinking too much? – kosa Jul 09 '14 at 17:51
  • 1
    Yes. But thinking too much isn't such a bad thing. :) So: if the kinds of *differences* you are interested in testing for are larger (in terms of time) than the gaps in your data (e.g. you are interested in long terms trends), then you might be pretty safe. On the other hand, if you are interested in testing for 'spikiness' (very short term events), then it is possible that your missing data may utterly thwart you. – Alexis Jul 09 '14 at 17:54
  • We are more interested in trends than spikes. Based on our discussion it seems we can conclude that if we are using imputation to fill the gaps, then number of gaps in data set is ir-relevant subject to think about (assuming our motive is about future trending). – kosa Jul 09 '14 at 18:05
  • Hold on: the gaps are **not irrelevant to think about** the usual assumptions and concerns in multiple imputation still apply. It's more that inferences around your interests can be reasonably valid when imputing these kinds of missingness. – Alexis Jul 09 '14 at 18:30
  • Ok. Thanks! I think I need to do more reading on this multiple imputation concept to clear out some of the questions I have. Thanks! – kosa Jul 09 '14 at 18:38
0

If you have enough data to do a meaningful test, you could look at a chunk of the data with no missing values. Then remove some values and fill in the missing values with interpolations. Fit the Holt Winters model on the interpolated data, and look at the error of the model on a holdout section of your data to see how it compares to forecasting from the original data set. Then you can experiment with removing different numbers of values to see what kind of effect it has on the error.

BizForecaster
  • 321
  • 1
  • 6