Changepoints in R

Question

I have the following dataset:

results <- data.frame(Date = c("A", "B", "C", "D", "E", "F", "G", "H","I", "J", "K", "L", "M", "N", "O", "P", "Q", "R", "S"),
                  P1 = c(0.43, 0.45, 0.57, 0.15, 0.5, 0.33, 0.26, 0.81, 0.43, 0.48, 0.14, 0.26,-0.21, 0.27, 0.37, 0.33, 0.68, 0.15, 0.44))

I want to know, if there are statistically significant changes of my observations.

So I thought to use changepoint analysis.

First I used the bcp-Package, with the following code:

c <- bcp(results$P1)
plot(c) 

However, there are no changepoints according to this plot.

Then I used the "changepoint" package and the following code:

var=cpt.var(results$P1, method="PELT")
plot(var)

Here I get three possible changepoints, but not were I supposed them to be (for example not in M, but in N).

Can anybody explain me why? Or is there another way to show, if the values changed significantly from one observation to another?

Answer 1

IrishStat is correct in that you are trying to identify a change in mean, not a change in variance. Thus in the changepoint package you should be using mean=cpt.mean(results$P1, method="PELT") instead. As for the bcp package this gives no changes in mean.

The cpt.var function gave 3 changes in variance because the variances of each part, calculated using

segvar=param.est(var)$variance
segvar
>0.02126190 0.09944762 0.00080000 0.07043333

segvar[-1]/segvar[-length(segvar)]
>4.677267637  0.008044436 88.041666667

Typically changes in variance are detected with roughly 80% power or more if the ratio of neighbouring variances is greater than 3 (or less than 1/3). The ratio of these variances clearly fits this paradigm which is why the changes were detected but not necessarily in the places you would have expected to identify a change in mean.

Note that this is all based on a penalty that only penalizes the number of changepoints. This is why segment lengths of 2/3 observations are detected. If the application suggests segments of small lengths such as these are implausible then I would use a penalty that penalizes segment length too (or set a minimum segment length).

See introductory references at www.changepoint.info for more background details on changepoint analysis. There is also a list of various changepoint open source software packages there.

Answer 2

Change Point analysis can mean detecting parameter changes OR changes in error variance OR changes in the Expected Value. Your example is simply the latter. Detecting change points in the Expected Value is called Intervention Detection http://www.unc.edu/~jbhill/tsay.pdf . This has been significantly enhanced in a product called http://www.autobox.com/cms/ which I helped develop that even incorporates user-specified predict series. I took your 19 values and AUTOBOX detected change points at M,H, and Q (13th,8th and 17th values) Note well that in general ARIMA structure has to be taken into account in time series analysis. In this case there is no provable arima structure ( autoregressive memory). It appears to me that your free software has a cost.

Exploratory Data Analysis as championed by Tukey, Box et.al. speaks to extracting hypothesis/information from the actual data such as the form of the ARIMA model, the detection of anomalies , changes in parameters etc. all of which do not require specific quantitative hypothesis promulgated before the data arrives. If one is lucky enough to have these hypothesis handy one can of course test them. Modern analytics actually learn from the data. What you knew (visually or scientifically) to be unusual was indeed detectable. I would say that the Tsay reference is an example of a trustworthy procedure.