Sampling from a Gaussian distribution

Question

I am trying to generate samples from a Gaussian distribution, with the sample having a certain probability, but unable to figure out a way. Eg. I want a sample from $N(0, 1)$ which has a probability of $0.3$.

For context, I am trying approximate inference for a Bayesian network.

For discrete variables, I divide the real line between 0 and 1 at the intervals equal to the value of probability for each of the discrete value. $P(X=x_0)=0.35$, $P(X=x_1) =0.65$, using random number generator, if the generated number is $\le 0.35$, $X$ is instantiated to $x_0$ else to $x_1$.

Unable to figure out a way to do the same for continuous distribution.

Answer 1

Usually you would use a built-in function of your favourite package (R, numpy etc.) to do this. However, I suggest you first make sure you know what you want to sample from or at least restate your wish to sample from "N(0,1) with probability 0.3". N(0,1) is already a parametrised distribution, like the one you described for the discrete case. If you want to do the sampling manually, you would do so by reversing the process. I.e. draw a number from a uniform distribution and then look for which x the CDF of N(0,1) reaches the particular value you have drawn.

I hope that helps clarifying things a bit.

Answer 2

With sampling from continuous distributions, like normal, you can follow exactly the same procedure like in your discrete example. First, using cumulative distribution function $F_X$ you find probability that $X \le x$,

$$ \Pr(X \le x) = F_X(X) $$

and then using inverse of cumulative distribution function (so called, quantile function) you follow the procedure described by you. So first you sample $U$ that is uniformly distributed in $[0,1]$ and then take

$$ X = F_X^{-1}(U) $$

in your case it is quantile function of standard normal

$$ X = \Phi^{-1}(U) $$

or in R code:

U <- runif(1000)
X <- qnorm(U)

See this thread for more details: How does the inverse transform method work?

There are also other algorithms for sampling from normal distribution, as described, for example, in Wikipedia.