0

I'm using GPML (Gaussian Processes for Machine Learning) v3.2 in Matlab.

meanfunc = @meanConst; hyp.mean = 0;
covfunc = @covSEard; ell = 1.0; sf = 1.0; hyp.cov = log([ell*ones(1,size(xtr,2)) sf]);
likfunc = @likGauss; sn = .1; hyp.lik = log(sn);
inffunc = @infLOO;
hyp = minimize(hyp, @gp, -4*40, inffunc, meanfunc, covfunc, likfunc, xtr, ytr);
[myte vyte mfte vfte lp] = gp(hyp, inffunc, meanfunc, covfunc, likfunc, xtr, ytr, xte, yte);

exp(lp) has values greater than 1 (e.g., 1.1254).

Why?

COOLSerdash
  • 25,317
  • 8
  • 73
  • 123
qwer1304
  • 81
  • 1
  • 5

1 Answers1

3

This is a big aha moment when going from discrete to continuous random variables.

Probability densities in continuous spaces can be greater than 1. The restriction that they should be below 1 is only for discrete random variables.

For continuous random variables the integral of the density has to be one:

$$\int_xp(x)dx = 1,$$

Thus, a uniform distribution over the interval $[0, 0.1]$ has to have a probability density of $p(x) = 10$ everywhere. It is only intervals (and their sums) which will always be less than 1.

bayerj
  • 12,735
  • 3
  • 35
  • 56
  • It's _densities_ that can be greater than 1, which is your main point. It is never true that probabilities can exceed 1, so the answer needs editing. – Nick Cox Jun 21 '13 at 11:28
  • Well, I know that DENSITY could be > 1, but it's the PROBABILITY which is > 1 in this case. So the question stands. – qwer1304 Jun 21 '13 at 20:44
  • No. Check the book/docs/math, it is the value of a probability density function. (It is confusing, since Rasmussen's bookand the GPML package use the word probability as loosely as I did in my first answer.) – bayerj Jun 22 '13 at 19:06