3

I understand from wikipedia that a variogram model must be positive definite to be used for kriging:

Note that the experimental variogram is an empirical estimate of the covariance of a Gaussian process. As such, it may not be positive definite and hence not directly usable in kriging, without constraints or further processing. This explains why only a limited number of variogram models are used: most commonly, the linear, the spherical, the Gaussian and the exponential models.

...But, I don't understand why the variogram model must be positive definite?

I know that covariance matrices are always positive semi-definite, so I might be able to understand why a positivesemi-definite matrix is necessary. But, why positive definite?

I'm having trouble tying these concepts together: kriging, positive definite matrix, covariance, variogram.

Danica
  • 21,852
  • 1
  • 59
  • 115
makansij
  • 1,919
  • 5
  • 27
  • 38

1 Answers1

2

The terminology about positive definite matrices is very inconsistent. Some authors use "positive definite" to mean all-positive eigenvalues and "positive semidefinite" to mean all-nonnegative eigenvalues; some use "positive definite" to mean all-nonnegative eigenvalues and "strictly positive definite" to mean all-positive. Here, I think the latter use is meant: to be able to perform the process of kriging, the matrix must be positive semidefinite.

That said, kriging with singular covariance matrices is mildly unusual, and some code / default ways of thinking about the process might be broken by it.

Danica
  • 21,852
  • 1
  • 59
  • 115
  • okay, thanks. But what is implied by an eigenvalue of 0 in a covariance matrix (since that's what `positive semi-definite` means)? – makansij Sep 18 '16 at 01:50
  • @Hunle Let $X \sim \mathcal{N}(\mu, \Sigma)$; an eigenvalue of zero means that there is some $v$ with $\Sigma v = 0$. But that means that if we observe along the direction $v$, i.e. we take $v X$, then that has $\mathrm{Var}[v X] = v^T \mathrm{Var}[X] v = v^T \Sigma v = 0$. So an eigenvalue of 0 means that $v X$ is known exactly. – Danica Sep 18 '16 at 01:52
  • 1
    I do not fully understand these things, but I have seen codes to generate gridded Gaussian random fields via FFT that use ["circulant embedding"](https://scholar.google.com/scholar?q=circulant+embedding+gaussian+random+field+FFT), where in practice they truncate negative (eigen-)values to 0. – GeoMatt22 Sep 18 '16 at 01:56
  • @Dougal, I'm sorry, but I don't fully understand your comment. what is $v$ in your example? – makansij Sep 20 '16 at 06:38
  • Once we've seen a bunch of data, we have a probability distribution (a Gaussian process) of what everything should look like. Choose points to observe, and stack up their values in a random vector $X$. $X$ is distributed as a normal distribution with mean $\mu$ and variance $\Sigma$. If $\Sigma$ is singular, with a vector $v$ in its null space, then $v X$ will be known exactly. If $\Sigma$ were the zero matrix, then this is true for *any* vector $v$, so $X$ would be known exactly. If it had the ones vector in its null space, then we might not know each component of $X$ but we'd know its sum. – Danica Sep 20 '16 at 06:46