Is it valid to use an ARMAX model for TV Attribution?

Question

Suppose I have a website which has some baseline hourly traffic. I also run TV advertising intermittently which drives up my web traffic. I want to determine how much effect my TV advertising is having in terms of driving up web traffic.

If I fit an ARMAX model with hourly TV advertising spend or impressions as exogenous variables, is it valid to claim that the AR terms represent the "baseline traffic" while the regression terms represent the traffic that should be attributed to TV advertising?

Here is some example code of what I'm trying to do:

library(forecast)

xmat <- as.matrix(cbind(data[,c("AdSpend","Impressions")]))
xvar <- data$WebSessions

fit <- Arima(x=xvar, xreg=xmat, order=c(12,0,0), include.constant=FALSE)

reg_terms <- fit$coef["AdSpend"] * data$AdSpend + fit$coef["Impressions"] * data$Impressions
AR_terms <- fitted(fit) - reg_terms

I can then create a stacked area chart using AR_terms (the baseline hourly web traffic) and reg_terms (the TV attributed hourly traffic).

Is this a valid approach?

Thanks for the help.

Answer 1

This is an excellent question. I recommend that you get a cup of coffee and carefully read through Rob Hyndman's blog post on "The ARIMAX model muddle".

Basically, the answer is no. If you fit a straightforward AR(I)MAX model, your covariate coefficients cannot be interpreted as the promotion effect. The problem is that a change in a covariate value will have an effect on the forecast that depends on prior fits. This is very hard to interpret and to communicate.

However, not all is lost, because your call to R's Arima() does not, in fact, fit an AR(I)MAX model. Rather, it first regresses your observations on the covariates, and then models the residuals with an AR(I)MA process. That is, it fits a so-called regression with AR(I)MA errors. And for this model, your interpretation - the covariates and their coefficients capture promotional effects, while the ARMA part captures "the rest" - is perfectly valid.

Now, whether an AR(I)MAX model or a regression with AR(I)MA errors produces better forecasts I don't know. Given that I don't know an easy way to actually fit an AR(I)MAX model in R and the interpretational difficulties described above, I'd recommend that you don't worry overly over true AR(I)MAX models and stick with what Arima() gives you.

However, I suspect that web surfing and TV watching may have multiple-seasonalities - the intra-day patterns may well be different between the weekend and the rest of the week. I don't see this in your plot, but you may want to look whether your data exhibit this. If so, there has been some work on forecasting with multiple seasonalities, mostly using variants of Exponential Smoothing and/or State Space Models. Some of these may be able to simultaneously model multiple seasonalities and explanatory variables.