6

The Slice sampler1 takes as its argument the log of the density to be sampled from.

Why is it doing this? A commenter on this question pointed out that it makes no sense to "sample" from the log of the density, since it is often a negative value. Why then, does the Slice sampler take the log of the density, and how does it still work?

Code to Radford M. Neal's Slice sampler, on his website: http://www.cs.toronto.edu/~radford/ftp/slice-R-prog

1Neal, Radford M: Slice sampling, Annals of Statistics 31(3), 705–741, 2003

Community
  • 1
dwcoder
  • 338
  • 2
  • 9
  • If you multiply enough densities together the number of digits used to represent the result gets too large and cannot be represented by the computer. This problem is resolved by taking the logs and adding those. – Livid Feb 15 '15 at 01:00

1 Answers1

7

The slice sampler does not "sample from the log-density". It can, however, use the log density in the calculations to obtain a dependent sequence of observations from the density.

The basic idea of a slice sampler is in terms of the density itself, but for various reasons (computational accuracy, primarily) it's usually more convenient to work with the log-density.

As long as you use do everything in such a way as to be completely consistent with doing it in terms of the density, there's no problem whatever working with the log density.

The description for the univariate case in wikipedia here says:

  • Given a sample $x$ we choose $y$ uniformly at random from the interval $[0, f(x)]$;
  • given $y$ we choose $x$ uniformly at random from the set $f^{-1}[y, +\infty)$.
  • The sample of $x$ is obtained by ignoring the $y$ values.

That first step can for example be replaced by sampling from a negative exponential distribution in $-\log y$ where the lower bound of the exponential is at $-\log(f(x))$; the rest of the algorithm (with the obvious change to accommodate the fact that we have a log scale $y$) proceeds much as before. The result is exactly as if we had accurately sampled according to the original algorithm, which is cast in terms of sampling uniformly under the density itself at $x$ ... but in practice the calculation on the log-scale does that job "better"- with less accuracy loss.

Neal explains the motivation for this quite clearly in the Annals paper you included a reference for (p712):

In practice, it is often safer to compute $g(x)=\log(f(x))$ rather than $f(x)$ itself, in order to avoid possible problems with floating-point underflow. One can then use the auxiliary variable $z=\log(y)=g(x_0)−e$, where $e$ is exponentially distributed with mean one, and define the slice by $S={x:z<g(x)}$.

Glen_b
  • 257,508
  • 32
  • 553
  • 939
  • Great answer, thank you. I thought it was about the floating get point underflow, and hence I asked the other question about the MH using the log(density) to sample from the density. But it was described as nonsensical to want to do that, despite the ARMS and slice sampler using this "trick". I was having floating point underflow problems in one of my algorithms. – dwcoder Feb 15 '15 at 07:31
  • 1
    If you construct it correctly -- so you're still sampling from the density, you can do the calculations on the log scale (and work with log-density) with MH as well. – Glen_b Feb 15 '15 at 07:38
  • 1
    Exactly, that was what I meant with my other question. Do you suggest I change it's wording to make it clearer? – dwcoder Feb 15 '15 at 07:39
  • Indeed, it may help if you show that it's not sampling the log-density – Glen_b Feb 15 '15 at 07:41