6

From what I understand, SIR is a mechanism for sampling from a distribution $p$ that works as follows:

  1. Approximate a target distribution $p$ using an importance sample $S$ from a proposal distribution $q$
  2. Draw a small sample $S_\text{small}$ from $S$ with replacement according to $p$

The result, $S_\text{small}$, is a sample from $p$.

  1. Why use a small subset and not the full sample $S$ in 1?
  2. Why is this a variance reduction technique, and why does it matter to reduce the variance?
Amelio Vazquez-Reina
  • 17,546
  • 26
  • 74
  • 110
  • The resulting sample $S_\text{small}$ is _not_ a sample from $p$ due to the sampling from self-normalised weights. – Xi'an Nov 25 '19 at 06:14
  • @Xi'an would you please elaborate the reason why $S_{small}$ is not a set of samples from $p$? Thanks – True Jul 12 '21 at 15:31
  • 1
    @True: dividing the importance weights by the sum of the importance weights modifies or biases the distribution of the resulting sample. – Xi'an Jul 12 '21 at 17:24
  • @Xi'an I see. Thanks a lot! :) – True Jul 13 '21 at 20:56

1 Answers1

2

The main point is that most of the time, SIS/SIR is used in a sequential setting and this is why one needs to reallocate particles to best deal with the next time integration. When performing IS, it is quite common that all of the weightings are attributed to only a very small subpart of the particles whereas they characterise the area of interest. Resampling allows reallocating particles from low-density regions into high-density regions making thus a more optimal use of our available particles (because the more particles, the more costly the procedure).

As I understand it, $S_\rm{small}$ is not smaller than $S$: they have typically the same size.

gung - Reinstate Monica
  • 132,789
  • 81
  • 357
  • 650
peuhp
  • 4,622
  • 20
  • 38