1

Definition of a GP: a stochastic process $\{X_t, t\in T\}$ such that for every finite set of realization times $t_1, \dots, t_k \in T$, the joint $(X_{t_1}, X_{t_2}, \dots, X_{t_k})$ is a multivariate Gaussian random variable.

I am wondering why we don't simply define a GP to be $\{X_t, t\in T\}$ such that for every $t$, $X_t$ is a single-variate Gaussian variable. If this were the case, then $X_t$ would satisfy the above definition (since a joint distribution of Gaussians is a multivariate Gaussian).

I'm guessing this second definition is broader than the true definition. Are there any processes $X_t$ that satisfy the definition I suggested but isn't an actual GP?

Thanks.

900edges
  • 339
  • 1
  • 9
  • 3
    Re "a joint distribution of Gaussians is a multivariate Gaussian:" please see https://stats.stackexchange.com/a/30205/919. – whuber Feb 24 '21 at 17:54
  • Dang, that is a major counterargument, thank you! So it seems that a joint is Gaussian if the marginals are independent and gaussian. But that would only form a joint Gaussian with 0 covariances (i.e. that condition is sufficient but not necessary), right? Do you know what requirements would allow a set of gaussians to give a joint Gaussian with nonzero covariances? – 900edges Feb 24 '21 at 18:16
  • See https://stats.stackexchange.com/questions/4364. Note that a joint Gaussian with nonzero covariances is *always* a linear transformation of a vector of independent Gaussian variables. – whuber Feb 24 '21 at 18:23
  • Can you clarify which part to look at? – 900edges Feb 24 '21 at 18:28
  • All of the answers! – whuber Feb 24 '21 at 19:06

3 Answers3

6

Yes, it is possible to construct processes which are marginally normally distributed but not jointly normally distributed.

Take for example two variables that are not jointly normal distributed: $$ X\sim N(0,1)\\ Y=|U|\cdot \text{sgn}(X), \quad U\sim N(0,1) $$ and construct the stochastic process $$ X_t=\begin{cases} X \quad \text{if } t<1\\ Y \quad \text{if } t\geq 1 \end{cases} $$

The variable $X_t$ is gaussian distributed for any $t$ but $(X_{0.5},X_2)$ is not gaussian distributed.

svendvn
  • 668
  • 3
  • 13
3

One of the most important features of GPs is the relationship between its output values, which is characterized by a kernel function $k(x,y) = \mathrm{Cov}(f(x),f(y))$.

If you define $f(x)$ only through marginals as $f(x) \sim \mathcal{N}(\mu(x), \sigma(x))$, you do not define any relationship between $f(x), f(y)$ for $x\neq y$.

ArnoV
  • 1,170
  • 3
  • 8
  • I see -- but whether or not we know the covariance has no effect on the definition of a GP. (All we know is that the joint is gaussian distributed, but no requirement on knowing the covariance matrix). – 900edges Feb 24 '21 at 16:43
1

For my reference and anyone else who might want it:

The best general answer results from whuber's referred post, showing that Gaussian marginals do not generally combine to form a Gaussian joint. This shows that the answer to my posed question is no, in general (not just if variables are discontinuous, as demonstrated by svendvn).

The case where marginal Gaussians do form a Gaussian joint is when the variables are independent or are independent under some linear transformation: as whuber mentioned, any multivariate Gaussian (MVG) is equivalent under linear transformation to a MVG with zero covariances (*see below for explanation).

Another characterization is that the collection of rvs $X_1, \dots, X_n$ is jointly Gaussian iff for any $a_1, \dots, a_n$, the linear combination $\sum_i a_i X_i$ is Gaussian (see here).

*If we have a multivariate $X\sim \mathcal{N}(\mu, \Sigma)$, then a linear transformation $A$ gives $AX \sim \mathcal{N}(A\mu, A\Sigma A^\top)$. Since any covariance matrix $C$ is symmetric, it has the diagonalization $C = Q\Lambda Q^\top$ where $\Lambda$ is diagonal and $Q$ is an orthogonal transformation. This means that our $X$ can be transformed orthogonally to a coordinate system where it is a joint distribution of independent Gaussian marginals.

So in summary, it is necessary to define a GP as having joint Gaussian rather than joint marginals, and an alternate definition along this second route would be require an additional requirement of independence, i.e. "a stochastic process $X_t$ is a GP if for every $t$, $X_t$ is Gaussian distributed and every linear combination of a subset $(X_{t_1}, \dots, X_{t_n})$ is Gaussian.

Note: I'm still not sure how to prove that a set of variables is independent under some linear transformation

900edges
  • 339
  • 1
  • 9
  • I am not sure what you mean by "discontinuous" in relation to the example in my answer. Both the X and Y are continuous random variables and (X,Y) is a continuous random vector. Perhaps you are thinking about the density function of (X,Y) or perhaps X_t as a function of t? neither of those are continuous. – svendvn Mar 09 '21 at 09:18