Comparing Coefficients of Two Time Series Models

Question

I am estimating a time series model. For simplicity, let it be:

$y_t=\beta_0+\beta_1y_{t-1}+\epsilon_t$.

I estimate this model for a whole panel of $n$ sections, i.e. I end up with a tuple of coefficients $\beta_{1,1},...,\beta_{1,n}$.

I also have a slightly adjusted version of the same panel. The adjustment is not of statistical nature. I am dealing with accouning data and the adjusted version is simply some accounting adjustment that is being introduced. More specifically, the adjustments occur at specific points in time, i.e. not every quantity of a time series is adjusted. I again estimate a time series model:

$p_t=\gamma_0+\gamma_1p_{t-1}+\omega_t$,

where $p$ denotes the adjusted $y$ and $k$ denotes the adjusted $x$. I thus get a tuple of coefficients $\gamma_{1,1},...,\gamma_{1,n}$.

I now want to test wether the independent variable has a significantly larger effect on the dependent variable in the adjusted panel than in the unadjusted panel, i.e. I want to compare the persistence (coefficient of an AR1 model) of the two panels. How do I do this? That is, how do I determine which coefficient of the two models applied to different sets of data is of significantly higher value.

Specifically, two issues have to be considered:

(1) I dont have two values which I want to compare. I have two tuples of values. I was thinking about a decision threshhold. For example, if more than half of the $\beta_{1,1},...,\beta_{1,n}$ are higher than their adjusted counterpart $\gamma_{1,1},...,\gamma_{1,n}$, the effect of the independent variable on the dependent variable seems to be higher in the unadjusted dataset. Is this reasonable?

(2) I need to test wether the the proposed difference between the coefficients is significant. How do I test wether the regression coefficients from two models applied to different data are significantly different? Is the Chow-Test appropriate here?

How do I tackle these issues at once?

Answer 1

I fully concur with the last paragraph of @AlexC-L's answer which is in essence a paired comparisons method. I have a feeling, though, that you do not want to look at the raw differences $\delta_i = \beta_i - \gamma_i$. The $\beta_i$ and $\gamma_i$ are presumably estimated by regression and are affected by uncertainty: does $\hat\beta_i=0.8$ with standard deviation of 0.6 imply higher persistence that $\hat\gamma_i=0.5$ with standard deviation 0.1? I would think the second more indicative of persistence than the first, which is not even significantly different from zero.

The choice depends on your problem, but I think you might at least consider to take not the raw estimated coefficients, but rather the coefficients measured in standard deviations when you compute the differences.

Answer 2

I assume when saying "test wether the independent variable has a significantly larger effect on the dependent variable in the adjusted panel than in the unadjusted panel", you are actually trying to find out which model can better describe the uncertainties and relations among the observations.

So instead of comparing the difference of the coefficients, a better approach is to perform model selection on your models. Since model selection has to be done on the same set of samples, you need to some how tweak your models to make them applying to the same sample set:

Model 1: $$ \begin{align}y & = \beta_0 + \beta_1x_1 + ... + \beta_n x_n + \epsilon \\\Rightarrow y &\sim F(y|x_{1:n},\theta_1) \\\end{align} $$ Where $\theta_1 = \{\beta_{0:n} \text{ and all the other parameters}\}$, you can understand $F(y|x_{1:n},\theta_1)$ as a distribution of $y$ conditioned on $(x_{1:n},\theta_1)$. For example when the model is a simple linear regression $y = \beta_0 + \beta_1x_1 + ... + \beta_n x_n + \epsilon,\epsilon \sim N(0,\sigma^2)$, then $F(y|x_{1:n},\theta_1)$ will be a normal distribution with mean $\beta_0+\beta_1x_1+...+\beta_nx_n$ and variance $\sigma^2$, i.e. $F(y|x_{1:n},\theta_1) = N(y|\beta_0+\beta_1x_1+...+\beta_nx_n, \sigma^2)$

Model 2:

No matter how you "adjust" your samples, there must be a way to represent the adjustment with a function, say $p = h(y)$, $k=g(x)$. For example if the adjustment is discounting future payment $y$ into current value $p$, then the function $h(y)$ will be something like $h(y) = \frac{y}{(1+r)^m}$. With this idea in mind, your second model $p = \gamma_0 + \gamma_1k_1+,...,+\gamma_nk_n+\epsilon_2$ can be rewritten as: $$ \begin{align}y & = h^{-1}(\gamma_0 + \gamma_1g(x_1)+,...,+\gamma_ng(x_n)+\epsilon_2) \\\Rightarrow y &\sim G(y|x_{1:n},\theta_2) \\\end{align} $$ Where $\theta_2 = \{\gamma_{0:n} \text{ and all the other parameters involved in h() and g()}\}$.

Now that both the models are put to the same set of samples, you can start comparing them. There are two common ways to perform the comparison:

• method1: If $F()$ and $G()$ are Bayesian models, you can compare their marginal likelihood (the higher the better) or BIC (the lower the better).
• method2: Use cross validation and compare their cross validated expected-prediction-error.

Answer 3

Note: This answer does not take into consideration that $\beta_i$ and $\gamma_i$ are themselves estimated (thank @Turell for pointing that out). I make another try in another answer.

---

You have n $\beta_i$ and n $\gamma_i$ that you want to compare. If n is large enough, you might turn this problem into the comparison between two distributions.

You may use the Kolmogorov–Smirnov test to determine if those two distributions are significantly different from each other. If this is the case, graphical inspection may then be used to determine if beta are generally higher than the gamma. To go beyond this simple graphical inspection, you may look at the differences between the quantiles of two distributions following Doksum, and Wilcox.

One issue I notice yet: above, this would not compare $\beta_{i0}$ and $\gamma_{i0}$ one to one. This can be fixed by defining: $\delta = \beta - \gamma$, and by comparing $\delta$ with a distribution of zeros.

Answer 4

Let us define $\delta_i = \beta_{1,i} - \gamma_{1,i}$, with i indexing the samples. You would ideally regress $\delta_i$ over 1: $\delta_i = b.1 +\eta_i $. If I make no mistake, your question can indeed be rephrased:: What is the value of $b$? Is $b$ significantly different from 0?

However, as noticed by @F.Tusel, there is some uncertainty regarding $\delta_i$; this will bias upwards the variance associated with $b$, and this could cause your result to be (wrongly) non-significant. If your result is significantly different from zero, stop here as what is below would only increase significance.

If not: Having uncertainty on the value of the dependent variable is however a classical problem. A clear explanation on how to deal with it in classical cases can be found in [1].

Is it simple in your case? It probably depends. Are your samples i.i.d. ?

• If yes: the variance-covariance matrix of $\delta_i$ is simply the diagonal matrix with $\sigma_{\delta_i}$ on the diagonal. And $\sigma_{\delta_i}$ can itself be infered from the matrixes of variance-covariance of $\left(\array{\beta_{1,i} \\ \gamma_{1,i} }\right)$. The latter can be estimated if you jointly estimate: $$ \left(\array{y_{t,i} \\ p_{t,i}}\right) = \left(\array{\beta_{0,i} \\ \gamma_{0,i} }\right) + \left(\array{y_{t-1,i} \ \ 0 \\ 0 \ \ p_{t-1,i}}\right)\left(\array{\beta_{1,i} \\ \gamma_{1,i} }\right) + \left(\array{\epsilon_{t,i} \\ \omega_{t,i} }\right) $$.

• If not, more thinking is required. Please add more information in the Question regarding your samples.

---

Reference: [1] Lewis, Jeffrey B., and Drew A. Linzer. "Estimating regression models in which the dependent variable is based on estimates." Political analysis 13.4 (2005): 345-364. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.318.7018&rep=rep1&type=pdf