7

I've heard about the use of copulas to transform data. For instance, supposedly it's applied to data that is non-normal to make it look more normal. However, I don't quite understand how this is done. I've read introductions to copula's (e.g. Nelsen's "An Introduction to Copula's", among others), and although I understand the concept (at least I think I do), I only see copula's being used as a measure of dependence between two or more random variables. I also read questions such as this one and this one, but I still don't see it.

To give my problem a more concrete setting, I have a dataset consisting of multivariate time series, $\{X_t\}_{t=1}^{T}$ where $X_t \in \mathbb{R}^d$. I want to fit a model to this dataset that assumes that $$X_t | X_{t-1} \sim \mathcal{N}(\Gamma X_{t-1},\Omega)$$ (where $\Gamma$ is a $d \times d$ matrix) but my data does not seem to adhere to this assumption (at least, not according to multivariate normality tests in R). Can I use copulas to transform this dataset into a dataset that is more normal than the original one?

Stijn
  • 1,550
  • 1
  • 12
  • 20
  • 1
    What exactly do you mean by "more Normal"? Are you saying the *marginals* of your data do not appear to be Normal and you would like them to be? Or are you referring to the correlation structure of your data (and if so, how does one compare the "normalness" of different correlation matrices)? – whuber Nov 26 '14 at 21:34
  • 1
    @whuber: Maybe the intention is to get the dependency structure more like a normal copula? – Horst Grünbusch Nov 26 '14 at 22:09
  • 1
    @Horst Or maybe the intention is to make the marginals Normal. Or maybe both (in some sense). It's unclear. – whuber Nov 26 '14 at 22:11
  • 1
    Do you have one sample of $(X_t)$ in your data or several replicated samples ? – Stéphane Laurent Nov 30 '14 at 20:30
  • 2
    "*supposedly it's applied to data that is non-normal to make it look more normal*" -- can you show someone actually saying this? Some context might help. Are your sources talking about joint, conditional or marginal distributions being 'more normal'? – Glen_b Nov 30 '14 at 22:08

2 Answers2

3

This page by MathWorks has a detailed description of using copulas for various tasks with a lot of examples within MATLAB: Probability Distributions Used for Multivariate Modeling. This is helpful for just seeing the nuts and bolts of how copulas can be used in some simple cases.

For transformation of a data matrix $X$ to the state of having marginal standard normal distribution functions, there is essentially a two-step process.

The first step is to transform the margins of the data to the uniform distribution. This can be done using a fit to a theoretically known distribution, using the empirical distribution function, or using a smooth estimator of the distribution function.

The second step is to use the quantile function of the normal distribution function to transform the margins of the data to normality.

This is the mathematical theory. I don't know right now what the ramifications are of using mis-specified distributions or estimated distributions in place of the theoretical distributions.

The R package regpro has a built-in function called copula.trans() that transforms your data to have margins that are distributed as standard normals. In other words, it carries out the two steps described. If X is your $n\times d$ data matrix, then copula.trans(X) gives you back a data matrix with marginals transformed to theoretically have standard normal distributions (by default).

How to get back is not included in the package. To back-track the process, you would first need the percentile function for the standard normal, then second your theoretical distribution function, your interpolated empirical distribution function, or your smooth estimator of the distribution function.

It would be nice if having marginal normal distributions would be sufficient to drive at least multivariate normality after this process. However, that is unfortunately not generally true, as some of us know. For a simple counter-example, see: Two normally distributed random variables need not be jointly bivariate normal. (See @Glen_b's comment below for a bit more information.)

Even aside from that, it looks like some further assumption might be needed to assure that the distribution after transformation would follow a relationship like $X_t|X_{t-1} \sim {\cal{N}}(\Gamma X_{t-1}, \Omega)$.

Initially, I was looking at some results on conditional distributions that might provide some assurances such as those found in (Arnold and Pourahmadi, 1988) or (Ashsanullah and Wesolowski, 1994). An example is the exchangeability criterion $$ (X_1,...,X_{t-1}) \stackrel{d}{=} (X_2, ...,X_t).$$ However, the set-up is a bit different here. (See @Stéphane Lauren's comment below.)

Can I use copulas to transform this dataset into a dataset that is more normal than the original one?

Maybe using the copula transformation will work fine for what you have in mind, but there seems to be little theoretical underpinning to go the whole distance to the model you want to fit.

M. Ahsanullah and J. Wesolowski (1994) Multivariate normality via conditional normality. Statistics and Probability Letters. 20: 235--238.

B.C. Arnold and M. Pourahmadi (1988) Conditional characterizations of multivariate distributions. Metrika 35(1):95--108.

jvbraun
  • 2,239
  • 13
  • 19
  • 3
    This sentence concerns me: "*Theory says that having marginal normal distributions is necessary and sufficient to drive multivariate normality.*" -- without some additional conditions (having a Gaussian copula), this isn't true. Was it your intent that the additional conditions apply there? Or did you intend something other than 'marginal' in the sentence? – Glen_b Nov 30 '14 at 20:17
  • 2
    $X_2$ has possibly higher variance than $X_1$ for some choices of $\Gamma$, hence it is not true that the equality $(X_1,...,X_{t-1}) \stackrel{d}{=} (X_2, ...,X_t)$ is necessary for the model assumption (it is necessary wrong in such cases). – Stéphane Laurent Nov 30 '14 at 20:19
  • 2
    @Glen_b If by "concerns" you mean "Wrong, bucko," then I agree with you after reviewing it... Not sure *what* I was thinking. @ Stéphane Lauren – jvbraun Nov 30 '14 at 20:40
  • @Stéphane Lauren That puts a damper on that thought, too. My main thought was that even if one could bring the distribution to normality, that assuring that the conditional relationships would hold is another matter. Does that seem reasonable? – jvbraun Nov 30 '14 at 20:46
  • 2
    jvbraun - I didn't want to suggest anything stronger until your real intent was clearer, since it might easily have been a typographical error or a simple omission of a few words that didn't convey what you wanted. To add to your useful link in the edit, there are a number of examples on CV, including the ones [here](http://stats.stackexchange.com/questions/30159/is-it-possible-to-have-a-pair-of-gaussian-random-variables-for-which-the-joint-d/30205#30205) and [here](http://stats.stackexchange.com/questions/125648/transformation-chi-squared-to-normal-distribution/125653#125653) – Glen_b Nov 30 '14 at 21:59
3

Copulas used to recover the joint distribution from marginal distributions. This application is based on Sklar's theorem, which states that if you have two marginal distributions then they're linked through a copula to a joint distribution. A copula is simply a function itself. If you have a bi-variate distribution, then a copula is a bi-variate function too. What this theorem doesn't tell you is how to find this copula. It only tells you that it exists.

Coming to your question:

Can I use copulas to transform this dataset into a dataset that is more normal than the original one?

No. You can't. You stated

I want to fit a model to this dataset that assumes that $X_t|X_{t−1}\sim \mathcal{N}(ΓX_{t−1},Ω)$ (where $Γ$ is a d×d matrix) but my data does not seem to adhere to this assumption

That is the main issue: you want to fit the model, which doesn't seem to be an appropriate model for your data. If you really think that your data is not normal, then you should not be fitting a normal model. Period.

Now, you can use copulas to model your data in a different way though. Here's how.

  1. Pick a copula. How? You can start with Gaussian copula. The "nice" thing about this one is that if your marginal were, in fact, Gaussians, then the joint will be multivariate Gaussian. If your marginal are "kind of" normal, then this copula may work well for you.
  2. Build an empirical distribution, e.g. using ecdf in MATLAB. The idea is to feed your data into this function, which will build cumulative distribution without using any assumptions about the functional form of the distribution. Alternatively, you could use nonparametric distributions, such as kernel density estimations. The bottom line is to get the univariate cumulative distributions, i.e. marginals, because it's the input into copula.
  3. Fit the copula to data.
  4. Generate random sample using the fitted copula, it'll produce multivariate random numbers between 0 and 1.
  5. Plug the inverse empirical CDFs, to convert [0,1] ranges into the random variables.

Note, that the main assumption here is the copula. There are many copulas out there, you can try several of them. This is the weakest link in this chain.

Here's an example in MATLAB.

Aksakal
  • 55,939
  • 5
  • 90
  • 176
  • 1
    I'm afraid he only has one sample of $(X_t)$, because this is the classical data for a time series $X_t = \Gamma X_{t-1}+\epsilon_t$. If this is the case then there's no available ecdf. – Stéphane Laurent Dec 01 '14 at 09:10
  • 1
    You only need one sample for ecdf – Aksakal Dec 01 '14 at 12:38
  • 1
    I mean the data are $(x_1, \ldots, x_T)$ (a "trajectory" of the time series). There's no available ecdf of $X_1$ because there's only one observation of $X_1$ (namely $x_1$). – Stéphane Laurent Dec 01 '14 at 13:21
  • 1
    He assumes that the innovations are iid, so you can apply usual techniques such as copula under ergodic hypothesis. The only problem it seems that distribution is not normal, so I was suggesting ecdf. – Aksakal Dec 01 '14 at 13:27
  • 1
    ecdf of what ? The innovations are not observable. – Stéphane Laurent Dec 01 '14 at 16:06
  • 1
    Innovations are not observable, but residuals are. The simplest case is $r_t=X_t-X_{t-1}$. – Aksakal Dec 01 '14 at 17:12
  • 1
    Your $r_t$ is $(\Gamma-1)X_{t-1} + \epsilon_t$, I don't see what to do you with these residuals. The discussion becomes too extended. I just wanted to say that the OP question is not clear enough. – Stéphane Laurent Dec 01 '14 at 19:08