1

I am working on creating a distributed lag model between monthly NDVI and rainfall values. The data contains a stack of monthly values from 1982 - 2013, and I have to create the lag model for any one pixel in the dataset (in this case [170,200,] (see code block). I have been trying to remove the trend and seasonality from the data but it hasn't worked thus far. 

filename<-"E:/Semester 2/TSA/7/ndvi_1982_2013_sub"
filename2<-"E:/Semester 2/TSA/7/rain_1982_2013_sub"

ndvi<- read.ENVI(filename, headerfile=paste(filename, ".hdr", sep=""))
rain<- read.ENVI(filename2, headerfile=paste(filename, ".hdr", sep=""))

datan <- ts(ndvi[170,200,],start=c(1982,1),frequency=12)
datar <- ts(rain[170,200,],start=c(1982,1),frequency=12)

model <- lm(datan ~ datar) 
##here i model the original data just to look at 
##the ACF and PACF structure

par(mfrow=c(1,2))
acf(model$residuals,main="")
acf(model$residuals,type="p",main="")

ACF and PACF of the first regression model, clear seasonality

ACF and PACF of the first regression model, clear seasonality, so I used stl to remove the seasonal and trend components

decompdatan <- stl(datan,"periodic")
decompdatar <- stl(datar,"periodic")
acf(decompdatan[[1]][,3],main="")
acf(decompdatan[[1]][,3],type='p',main="")

Here the ACF and PACF of the residuals. Although the seasonal component has been removed there is still an apparent seasonal flucatuation

Here the ACF and PACF of the residuals. Although the seasonal component has been removed there is still an apparent seasonal flucatuation.

According to the assignment, I am supposed to move on to the pre-whitening stage after this step in order to determine the number of lags I should incorporate into the model, but the data is clearly not ready for this stage yet since there are some many significantly correlated lags still apparent in the data. Also, when I tried to iterate the number of lags within the arima model (for the pre-whitening), the AIC kept improving well past the orders of 8 and 9, but I have been told that the AIC should normally decrease until it reaches a sort of plateau, after which AIC values should again increase, thus making a clear choice of which order is most suitable.

I have tried other ways of removing the seasonal component such as using a different function and trying to create my own seasonal component using Fourier polynomials, but nothing seems to work. Is there something glaring I am doing wrong here?

Thanks.

Steve Ahlswede
  • 133
  • 3

1 Answers1

0

Often one needs to incorporate seasonal dummies as suggested by

If I am convinced that a series is mostly trend+season, what is it I should check about the residuals?

http://faculty.chicagobooth.edu/ruey.tsay/teaching/uts/lec10-08.pdf

Forecasting weekly demand: based on ACF and PACF, is ARIMA appropriate?

If you post your data or load it to dropbox , I will try and help further.

AFTER RECEIPT OF DATA:

You sent 384 monthly values for two time series. Here is the plot of the dependent series NVDI enter image description here . The general approach is called ARMAX modelling enter image description here

There are a number of issues to be identified via analytics:

  1. Is there seasonality (annual repetitive feaures) and if so is the seasonal structure autoregressive (SARIMA)or seasonally deterministic ( i.e the need for seasonal dummies )

  2. What differencing or de-meaning is necessary

3.) what is the form of the ARIMA structure

4) Are there unuusal values that need to be dealt with ( THE I'S)

5) Is there evidence of non-constant error variance or non-constant parameters over time in the residuals ( THE A'S )

6) What is the form of the relationship ( THE X'S )

Using AUTOBOX (a time series package that I have helped to develop ) I obtained answers to these questions in the form of a model that only contained statistically significant parameters and generated a plausible model.

enter image description here

The Actual/Fit and Forecast graph isenter image description here here and a less busy Actual and Forecast graph here enter image description here

The residuals appear to be quite free of structure enter image description here with ACF here ( suggesting a possible need for a slight MA(12) modification to the noise process)enter image description here .

The forecasts for the next 36 period are presented here enter image description here

The final model is developed through a sequence of interim models much like peeling an onion where model diagnostics ( tests of sufficiency and necessity ) are used to ultimately separate signal and noise . The Actual and Cleansed graph is informative about the identified anomalies enter image description here

In summary your attempt to limit your seasonal component to memory is at the root cause of your modelling worries.

IrishStat
  • 27,906
  • 5
  • 29
  • 55