2

I asked this question, Calculating the expression for the derivative of a Gaussian process, some time ago, and now I am interested in an extension to the question. So originally I wanted to know the following: Given $$f(x_1),f(x_2),....,f(x_n)\sim GP(X\beta,\sigma^2R)$$ where $R$ is the Gaussian correlation function

$$R=\exp\left\{-\sum_{i=1}^n\frac{|x_{ij}-x_{ik}|^2}{\phi_i}\right\}$$

then the distribution of $f'(x_1),f'(x_2),...,f'(x_n)$ is $$ f'(x_j) \sim \text{GP}\left( \beta_j, \sigma^2 \frac{2}{\phi_j} R\right) $$

Now what I would ultimately like to know is whether or not it is possible to calculate the posterior predictive process of the derivative process based on NON DERIVATIVE observations, i.e., is the following quantity defined?

$$f'(x_1),f'(x_2),...,f'(x_n)\,|\,\,f(x_1),f(x_2),...,f(x_n)\sim ???$$

@g g seems to suggest it can be done here: Derivative of a Gaussian Process but I don't follow the argument.

Community
  • 1
RustyStatistician
  • 1,709
  • 3
  • 13
  • 35
  • What is $f$? Usually if we know something distribution of $Y=f(X)$ it is hard to say something about distribution of $f'(X)$ if we do not know something specific about $g$ in $f'(x)=g(f(x))$. – mpiktas Apr 15 '16 at 13:23
  • I think this is a duplicate of the question linked in the last sentence (and the bounty should be put into the linked question to encourage more clear answers there) but apparently voting to close is impossible when there is an open bounty. In any case, the title here is somewhat misleading as this is about Gaussian processes specifically, not stochastic processes in general. – Juho Kokkala Apr 15 '16 at 13:26
  • @mpiktas $f$ is a Gaussian process (note that $x$ is not intended to be random, I'm not sure if your use of capital $X$ implies you misunderstood the question as being about evaluating some fixed function at a random $x$) – Juho Kokkala Apr 15 '16 at 13:28
  • @JuhoKokkala, from what I recall the Gaussian process is a stochastic process, whose fdds are Gaussian. And as far as I remember the derivative is not defined for such type of mathematical object. – mpiktas Apr 15 '16 at 13:33
  • @mpiktas, For example Rasmussen and Williams (see Eq. 4.4, http://www.gaussianprocess.org/gpml/chapters/RW4.pdf)) define the derivative in the mean square sense (this exists for some Gaussian processes, depending on the covariance function). I have no idea whether this implies existence of the derivative in some other sense (e.g. is there a version of the process for which the realizations are almost surely pathwise differentiable), but the present question seems valid in any case. See also discussion in the linked question. – Juho Kokkala Apr 15 '16 at 13:43
  • @JuhoKokkala, I've read up about this. Yes it is possible to define derivative in such setting, albeit it is quite specific, i.e. you need certain assumptions, which do not generalise well. In OPs case we have that $f$ is strictly defined, which makes the answer quite obvious and really Gaussian Processes are not needed here. – mpiktas Apr 15 '16 at 13:55
  • @mpiktas Where does OP say that $f$ is "strictly defined"? Are we reading the same question? – Juho Kokkala Apr 15 '16 at 14:04
  • @RustyStatistician can you clarify whether $f$ is intended to be a Gaussian process and $f'(x_k)$ the derivative of $f$ at $x_k$ (strictly speaking, the first equation here only says that $f$ at the test points is Gaussian but $f$ would need to be a Gaussian process 'everywhere' to get the derivatives. So it should perhaps be replaced with $f \sim GP$ etc. if that's indeed what is intended) – Juho Kokkala Apr 15 '16 at 14:10
  • @JuhoKokkala, according to OP, we have $Ef(x)=x\beta$. So we have a simple linear regression case. Isn't that the case? – mpiktas Apr 15 '16 at 14:11
  • @mpiktas Ok, now I see how we interpret this differently. Indeed, $Ef(x) = x\beta$ but (in my understanding) we are dealing with a Gaussian process whose mean is $x\beta$ and covariance function is as defined by $R$. The derivative in question is not the derivative of $Ef(x)$ but the derivative of $f(x)$, so it is not as simple as differentiating a linear regression. – Juho Kokkala Apr 15 '16 at 14:19
  • @JuhoKokkala yes $$f is intended to be a Gaussian process and $f′(x_k)$ the derivative of $f$ at $x_k$ – RustyStatistician Apr 15 '16 at 15:35
  • @mpiktas So essentially I want to know if I have a GP whose derivative is defined, is teh predictive process also defined if I don't have any observations of the derivatives? I.e., f'|f~ ??? – RustyStatistician Apr 15 '16 at 15:39

1 Answers1

0

I will consider the case of zero mean random process for simplicity. The generalization for nonzero mean is straightforward.

We need two facts from random processes theory:

  1. Covariance between random process and its' derivative is $$ \mathrm{cov}\left(f(x), \frac{\partial f(x')}{\partial x_i}\right) = \frac{\partial R(x, x')}{\partial x'_i}.$$
  2. Joint distribution of derivatives and values of Gaussian processes is multivariate normal.

The first fact allows you to calculate covariance matrix for joint of derivatives and values of a realization of Gaussian process $K$. Second fact allows you to get formula for marginal distribution of derivatives given values. Formula is similar to usual formula for Gaussian process $\hat{f}'(x) = k K^{-1} f$, but you need to insert specific covariance calculated with the formula given above.

Alexey Zaytsev
  • 2,335
  • 15
  • 26