18

I hope this is the right place to post this, I considered posting it on skeptics, but I figure they'd just say the study was statistically wrong. I'm curious about the flip side of the question which is how to do it right.

On the website Quantified Self, the author posted the results of an experiment of some metric of output measured on himself over time and compared before and after abruptly stopping drinking coffee. The results were evaluated subjectively and the author believed that he had evidence that there was a change in the time series and it was related to the change in the policy (drinking coffee)

What this reminds me of is models of the economy. We only have one economy (that we care about at the moment), so economists are often doing essentially n=1 experiments. The data is almost certainly autocorrelated over time because of this. The economists generally are watching, say the Fed, as it initiates a policy and trying to decide if the time series changed, potentially on account of the policy.

What is the appropriate test to determine if the time series has increase or decreased based on the data? How much data would I need? What tools exist? My initial googling suggest Markov Switching Time Series Models, but not my googling skills are failing me at helping do anything with just the name of the technique.

whuber
  • 281,159
  • 54
  • 637
  • 1,101
MatthewMartin
  • 280
  • 2
  • 6

6 Answers6

4

The Box-Tiao paper referred to by Jason was based on a known law change. The question here is how to detect the point in time. The answer is to use the Tsay procedure to detect Interventions be they Pulses, level Shifts , Seasonal Pulses and/or local time trends.

IrishStat
  • 27,906
  • 5
  • 29
  • 55
2

Josh said:

josh: From the OP "What is the appropriate test to determine if the time series has increase or decreased based on the data? ". This I believe asks for a determination if the mean of the residuals has shifted not the parameters of some ARIMA Model. In my opinion you are recommending the wrong software/solution procedure but that's just my opinion. – IrishStat Oct 28 '11 at 19:08

Suppose one starts with an AR(1) Model:

$$Y_t = \gamma + \phi*Y_{t-1} + E_t$$

Where ${E_t}$ is, say, a Gaussian Noise (mean zero and variance $\sigma^2$ The mean of this series.

The mean of the series is $\frac{\gamma}{1-phi}$

So, if for some time the parameters $\gamma$ and $\phi$ does not change, then so does the overall mean of the series. However, it any of these changes, necessarily the mean of the series will change. So, under piecewise stationarity, we are looking for changes of these parameters!

If structural models are assumed, Auto-PARM is the procedure to use.

Patrick Coulombe
  • 2,536
  • 6
  • 20
  • 28
user43737
  • 21
  • 1
1

Looking through some old notes on structural breaks, I have these two cites:

Enders, "Applied Econometric Time Series", 2nd edition, ch. 5.

Enders discusses interventions, pulse functions, gradual change functions, transfer functions, etc. This article may also be helpful:

Box, G.E.P. and G. C. Tiao. 1975. “Intervention Analysis with Applications to Economic and Environmental Problems.” Journal of the American Statistical Association 70: 70-79.

Jason Morgan
  • 1,164
  • 10
  • 15
1

Couldn't you just use a change point model, and then try to identify the change point using an MCMC algorithm such as Gibbs Sampling?

This should be relatively simple to implement, provided you have some prior distributions for your data or the full conditional distirbution (for Gibbs).

You can find a quick overview here

Berk U.
  • 4,265
  • 5
  • 21
  • 42
  • Just name-dropping the new `mcp` package which provides a high-level R interface to do change point detection using Gibbs sampling. It does autoregressive models too. [See more on the mcp website](https://lindeloev.github.io/). – Jonas Lindeløv Mar 06 '20 at 22:49
1

If you were considering all time points as candidate change points (a.k.a. break points, a.k.a. structural change) then the strucchange package is a very good option.

It seem that in your particular scenario, there is only one candidate time point. In this case, several quick options come to mind:

  1. T-test: a t-test on the hours of concentration per day on the "before quitting" vs. "after quitting" periods. If you are concerned with day-to-day correlation, you could give up some observations so that you have long enough intervals to believe the days are no longer correlated. With this approach,you will be trading off power with simplicity.
  2. AR: Fit an AR model with one dummy: "after quitting". If the predictor is significant, then you have a change. Using an AR, will capture the (possible) dependence between days.
JohnRos
  • 5,336
  • 26
  • 56
  • :John The idea is that you don't know the "one candidate time point" but want to find it analytically, perhaps for literally hundreds of time series. The "eye test" to determine this one candidate is often deficient as one-time pulses and underlying ARIMA structure occlude. Intervention Detection Methods a la R. Rsay or George Tiao searching for an unknown LEVEL/STEP shift actually constructs the variable you describe ( the one dummy with zeroes followed by 1's ) . Care should be taken to consider identifying the Interventions FIRST and then the ARIMA component and vice-versa. – IrishStat Oct 28 '11 at 17:56
  • @IrishStat: In the referenced Blog, the change point is known. For the cases it isn't, the strucchange R package was referenced. – JohnRos Oct 28 '11 at 19:47
  • :John From struchange documenation "Finally, the breakpoints in regression models with structural changes can be estimated" using the CHOW Method with which I am intimately familiar with.Testing or finding breakpoints in regression coefficients requires a specification of the regression model and if I am correct this has nothing whatsoever to do with answering the question"test to determine if the time series has increase or decreased based on the data?".I think that your recommendation is insufficient to answer the OP's question.Your recommendation answers a questionI don't believe was asked. – IrishStat Oct 28 '11 at 20:37
  • :john That is true but trivial as models with only an intercept are only found in textbooks or in dreams. – IrishStat Oct 29 '11 at 11:13
  • @IrishStat: it is true that structural-change framework is more general. Yet detecting an increase or decrease in the "data" can be done by fitting an intercept-only model. – JohnRos Oct 29 '11 at 11:16
  • :jOHN bUT without loss of generality it has to be done in concert with other structure such as ARIMA . What you have suggested is "Reductio ad absurdum" which is a form of argument in which a proposition is disproven by following its implications logically to an absurd consequence. – IrishStat Oct 29 '11 at 12:30
1

A few years ago I heard a talk by a grad student, Stacey Hancock, during a local ASA chapter meeting and it was on "structural break estimation" of time series. The talk was really interesting and I spoke with her afterwards and she was working with Richard Davis (of Brockwell-Davis), then at Colorado State University, now at Columbia. The talk was an extension of Davis et al. work in a 2006 JASA paper called Strutural Break Estimation for Nonstationary Time Series Models, which is freely available here.

Davis has a software implementation of the method that he calls Auto-PARM, which he made into a Windows executable. If you contact him you may be able to get a copy. I have a copy, and here is example output on a 1,200 observation time series:

    ============== RESULTS ===============
  ISLAND           1
    SC=   1910.58314770669
    Breaking point/AR order
           1              1
         351              1
         612              3
    ======================================
 Total time:   5.812500

So the series is AR(1) in the beginning, at observation 351 the AR(1) process changes to another AR(1) process (you can get the parameters), and then at observation 612 the process changes to AR(3).

One interesting setting I tried Auto-PARM on was looking at weekly ATM withdrawal data that was part of the NN5 competition. I recall the algorithm finding structural breaks in late November of a given year, e.g. the beginning of the US holiday shopping season.

So, how to use this algorithm via existing implementations? Well, again, you could reach out to Davis and see if you can get the Windows executable. When I was at Rogue Wave Software I worked with Davis to get Auto-PARM into the IMSL Numerical Libraries. The first language it was ported to was Fortran, where it is called Auto_PARM, and I suspect Rogue Wave will release a C port soon, with Python, C# and Java ports to follow.

Josh Hemann
  • 3,384
  • 1
  • 21
  • 18
  • :Josh he OP was not in my opinion referring to testing hypothesis of model parameter constancy , in your case where or not an AR(3) has constant parameters over time . He O believe is interesting in detecting a heretofore unknown shift in the mean of the residuals. – IrishStat Oct 28 '11 at 16:16
  • mods :The OP was not,in my opinion,referring to testing hypothesis of model parameter constancy,in your case whether or notan AR(3) has constant parameters over time.He I believe is interestedin detectinga heretofore unknown shift in the mean of the residuals.Thisis quite a different problem from the one you referred to.Now I totally agree that in the absence of Intervention Detection in the mean of the residuals.One might find a point in time where either the parametersof some model and/or thevariance of theerrors might have changed significantly BUT that's not whatthe OP wants to find out. – IrishStat Oct 28 '11 at 16:24
  • @IrishStat: Are you familiar with Auto-PARM? The algorithm uses residuals in the break estimation (both with respect to the number of breaks and the AR(p) order of the segments). The OP does not seems to have a specific method he is asking about. Rather, he seems to be asking very generally "If I am measuring a process in time and change something about the process, is there a way to detect this change point from the data alone?". He is not asking about level shift vs innovation vs additive outlier detection. Hopefully the OP can clarify for us... – Josh Hemann Oct 28 '11 at 18:56
  • josh: From the OP "What is the appropriate test to determine if the time series has increase or decreased based on the data? ". This I believe asks for a determination if the mean of the residuals has shifted not the parameters of some ARIMA Model. In my opinion you are recommending the wrong software/solution procedure but that's just my opinion. – IrishStat Oct 28 '11 at 19:08