12

Homework question:

Consider the 1-d Ising model.

Let $x = (x_1,...x_d)$. $x_i$ is either -1 or +1

$\pi(x) \propto e^{\sum_{i=1}^{39}x_ix_{i+1}}$

Design a gibbs sampling algorithm to generate samples approximately from target distribution $\pi(x)$.

My attempt:

Randomly choose values (either -1 or 1) to fill vector $x = (x_1,...x_{40})$. So maybe $x = (-1, -1, 1, 1, 1, -1, 1, 1,...,1)$. So this is $x^0$.

So now we need to move on and do the first iteration. We have to draw the 40 different x's for $x^1$ separately. So...

Draw $x_1^1$ from $\pi(x_1 | x_2^0,...,x_{40}^0)$

Draw $x_2^1$ from $\pi(x_2 | x_1^1, x_3^0,...,x_{40}^0)$

Draw $x_3^1$ from $\pi(x_3 | x_1^1, x_2^1, x_4^0,...,x_{40}^0)$

Etc..

So the part that's tripping me up is how do we actually draw from the conditional distribution. How does $\pi(x) \propto e^{\sum_{i=1}^{39}x_ix_{i+1}}$ come into play? Maybe an example of one draw would clear things up.

chl
  • 50,972
  • 18
  • 205
  • 364
Collin
  • 411
  • 4
  • 17

1 Answers1

11

Look at this case first. Dropping terms that do not depend on $x_1$, we have. $$ \pi(x_1\mid x_2,\dots,x_d) = \frac{\pi(x_1,x_2,\dots,x_d)}{\pi(x_2,\dots,x_d)} \propto e^{x_1 x_2} $$ $$ P(X_1=-1\mid X_2 = x_2, \dots, X_n=x_n) = \frac{e^{-x_2}}{C} $$ $$ P(X_1=1\mid X_2 = x_2, \dots, X_n=x_n) = \frac{e^{x_2}}{C} $$ $$ \frac{e^{-x_2}}{C} + \frac{e^{x_2}}{C} = 1 \Rightarrow C = 2 \cosh x_2 $$

x_1 <- sample(c(-1, 1), 1, prob = c(exp(-x_2), exp(x_2)) / (2*cosh(x_2)))

Generalize it to $x_2,\dots,x_{40}$ (take notice of the differences; see Ilmari's comment bellow).

Can you use Ising's analytic results to check your simulation?

Zen
  • 21,786
  • 3
  • 72
  • 114
  • So, it ends up only being dependent on the value immediately before it in the vector i.e. the only term that depends on $x_1$ is $x_2$, the only term that depends on $x_{23}$ is $x_{24}$, etc. What about the case of $x_{40}$? How do we draw it since the conditional distribution only seems to apply for $i=1$ through 39? – Collin Apr 23 '14 at 03:16
  • 1
    @user2079802: No, for $x_2$ through $x_{39}$ you get two terms in the exponent: $\pi(x_i\mid x_1,\dotsc,x_{i-1}; x_{i+1},\dotsc,x_d)$ $\propto$ $\exp(x_{i-1} x_i+x_i x_{i+1})$. But it's still easy enough to evaluate that for $x_i=\pm1$. – Ilmari Karonen Apr 23 '14 at 05:43