21

If $f_1,\ldots,f_k$ are known densities from which I can simulate, i.e., for which an algorithm is available. and if the product $$\prod_{i=1}^k f_i(x)^{\alpha_i}\qquad \alpha_1,\ldots,\alpha_k>0$$ is integrable, is there a generic approach to simulate from this product density using the simulators from the $f_i$'s?

Xi'an
  • 90,397
  • 9
  • 157
  • 575
  • 2
    Without additional assumptions, this seems unlikely. (Let $\alpha_i=1$ for simplicity. Let $\epsilon\gt 0$ be small. Suppose that associated with each $f_i$ is an interval $I_i$ on which $f_i\le 1$ and $\Pr_i(I_i)\gt 1-\epsilon$, outside of which $0\lt f_i\lt \epsilon$, and $I_i\cap I_j=\emptyset$ for $i\ne j$. Then the separate generators would almost always produce values in $I_i$, but the probability of $\prod f_i$ could be concentrated *anywhere,* seemingly unrelated to the $I_i$.) So, what else can you tell us about the $f_i$? – whuber Apr 10 '16 at 19:42
  • 1
    (+10) Correct! Using a smaller $\alpha_i$ would however lead to flatten all elements and hence favour overlap of their effective supports... – Xi'an Apr 11 '16 at 08:10
  • 1
    As whuber said tightness will be a problem, so I would take a transformation(OR preferential sampling) to cancel the tightness before generating random samples. There is one constructive approach I think I read a while ago. Sec 10.7 of https://link.springer.com/chapter/10.1007/978-1-4612-0209-7_10 Not sure if the discretization can also be applied here. – Henry.L Mar 14 '17 at 20:09

1 Answers1

4

Well, of course there's the acceptance-rejection algorithm, which I would implement for your example as:

  1. (Initialization) For each $i$, find $A_i = \sup_x \{\Pi_{j=1}^k f_j(x)^{\alpha_j}/f_i(x)\} $. Edit reflecting Xi'an's comment below: Select the distribution $f_i$ which corresponds to the smallest $A_i$.
  2. Generate $x$ from $f_i$.
  3. Calculate $\alpha = \Pi_{i=j}^k f_j(x)^{\alpha_j} / (A_if_i(x))$.
  4. Generate $u \sim U(0,1)$.
  5. If $u \leq \alpha$, return $x$, else go to 2.

Depending on the distributions, of course, you might have a very low acceptance rate. As it happens, the expected number of iterations is equal to the selected $A_i$ (assuming continuous distributions), so at least you are warned in advance.

jbowman
  • 31,550
  • 8
  • 54
  • 107
  • 4
    (+1) Indeed a solution! Assuming the bounds $A_i$ do exist for all $i$'s. Or even some $i$'s. Comparing the $A_i$'s [assuming they are finite] may also help in selecting the most efficient $f_i$. – Xi'an Sep 24 '17 at 13:16
  • 1
    I hadn't thought of it, but of course you're right, the $A_i$s themselves are very informative, as they are also equal to the expected number of iterations needed to actually generate the random number if you stick with one $i$ throughout. So you would want to pick the distribution with the smallest $A_i$ to use all the time. I'll edit the answer so your point doesn't get lost in comments. – jbowman Sep 24 '17 at 15:45
  • That is, assuming all $f_i$'s are properly normalised [as to integrate to one], which is not necessarily a standard occurrence. – Xi'an Sep 24 '17 at 16:57