5

Consider a sequence of $n$ independent Bernoulli trials drawn from a list of biases $p_1,p_2,...,p_n\in[0,1]$, respectively. We set the random variable $X$ to be the sum of these trials. On wikipedia, the distribution of $X$ is called the Poisson binomial distribution. We define the sample mean and sample variance of our list of Bernoulli biases as $$ \bar{p}=\frac{1}{N}\sum_{i=1}^n p_i $$ and $$ \sigma_p^2 =\frac{1}{N}\sum_{i=1}^N(p_i-\bar{p}) =\frac{1}{N}\sum_{i=1}^N p_i^2 - \bar{p}^2. $$

Since the trials are independent, it is easy to compute that $$ \mathbb{E}[X] = \sum_{i=1}^n p_i = N\bar{p} $$ and \begin{align*} \mathbb{Var}[X] &= \sum_{i=1}^n p_i(1-p_i) \\ &= N\bar{p} - N(\sigma_p^2+\bar{p}^2) \\ &= N\bar{p}(1-\bar{p}) - N\sigma_p^2. \end{align*}

The expectation value of $X$ is not surprising. Also, when $\sigma_p^2=0$ we must have $\bar{p}=p_1=\cdots=p_n$ and so $X$ is binomially distributed, which matches $\mathbb{Var}[X]$ computed above.

My confusion is this: why does the variance of $X$ go down as the sample variance $\sigma_p^2$ goes up (with $\bar{p}$ and $N$ fixed)? I find this very counter-intuitive, and would appreciate an explanation. I would expect with a greater variance of biases, there would be a broader distribution of possible sums of the result...

Ian Hincks
  • 417
  • 2
  • 8

1 Answers1

3

Think of the case where $n=2$. If $p_1 = p_2 = 0.5$, this maximizes the variance of X. If $p_1 = 0$ and $p_2 = 1$, then $X=1$ and there is no variance.

jlimahaverford
  • 3,535
  • 9
  • 23
  • Interesting. So you are saying, I think, that: the closer a given $p$ is to either 0 or 1, the more you know about the outcome of that trial. If $\sigma_p^2$ goes up, other quantities being fixed, then some of the $p$ values will be closer to either 0 or 1 (and somehow the ones that moved closer to $p=1/2$ have less of an effect than those that moved away from it?). Therefore you should be more certain about the total of the trials. – Ian Hincks Sep 30 '15 at 22:47
  • That is precisely the idea. You should be able to establish this by calculus, but since you had a formula that felt against your intuition I wanted to give you something simple. – jlimahaverford Sep 30 '15 at 23:50