6

Suppose $X_1,X_2,\ldots,X_n$ is a random sample from a Poisson Distribution with mean $\theta$. How can I find the conditional expectation $E \left( X_1+X_2+3X_3 |\sum_{i=1}^n X_i \right)$?

I know that $\sum X_i $ has a $poisson (n\theta$) distribution. Similarly the random variable $X_1+X_2+X_3$ has a $poisson (3\theta)$ distribution. I get confused with the required summations afterwards though.

Thank you.

JohnK
  • 18,298
  • 10
  • 60
  • 103

2 Answers2

4

Define $S_n=\sum_{i=1}^n X_i$. By symmetry, $$ \mathrm{E}\left[ X_1 \mid S_n \right] = \mathrm{E}\left[ X_2 \mid S_n \right] = \dots = \mathrm{E}\left[ X_n \mid S_n \right] \quad \textrm{a.s.} \quad (*) $$ Hence, using $(*)$ and the linearity of the conditional expectation, we have $$ \mathrm{E}\left[ X_1 \mid S_n \right] = \frac{1}{n} \mathrm{E}\left[ X_1+\dots+X_n \mid S_n \right] = \frac{1}{n} \mathrm{E}\left[ S_n \mid S_n \right] = \frac{S_n}{n} \quad \textrm{a.s.} $$ The same reasoning leads to $$ \mathrm{E}\left[ X_1 +X_2 +3X_3\mid S_n \right] = 5\,\mathrm{E}\left[ X_1 \mid S_n \right] = \frac{5\,S_n}{n} \quad \textrm{a.s.} $$ Now, remember that $S_n\sim \mathrm{Poisson}(n\theta)$, and find the pmf of $5\,S_n/n$ (consider its support).

Zen
  • 21,786
  • 3
  • 72
  • 114
  • Thank you. Can you please explain further what you have done in step 2? I do not understand why $E [X_1|S_n ] =S_n /n$ – JohnK Jan 19 '14 at 00:06
  • Can you use $(*)$ to prove that $\mathrm{E}\left[ X_1 \mid S_n \right] = \frac{1}{n} \mathrm{E}\left[ X_1+\dots+X_n \mid S_n \right]$? – Zen Jan 19 '14 at 00:09
  • Yep. It makes sense, thank you. I was asking about the $E[S_n|S_n]=S_n$. That is a cool identity. – JohnK Jan 19 '14 at 00:11
  • 1
    That's one of the properties of the conditional expectation. Informally, if you have the information $S_n$, then your best guess for $S_n$ is exactly $S_n$. – Zen Jan 19 '14 at 00:14
  • 2
    The important thing here is to understand that $\mathrm{E}[X]$ is a real number, while $\mathrm{E}[X\mid Y]$ is a random variable. Take alook at this question: http://stats.stackexchange.com/questions/38700/conditional-expectation-of-multivariate-distributions – Zen Jan 19 '14 at 00:15
  • Uh yeah ;). Enjoy your evening. – JohnK Jan 19 '14 at 00:15
  • I am working on a similar exercise and since your answer was helpful, I did not feel like starting another thread. What I wanted to ask you is whether it is right to say that $E[X_1 |\bar{X}=\bar{x}]=E[\bar{X}|\bar{x}]=\bar{x}$. I think it is but would you mind confirming that for me? Thank you. – JohnK Jan 28 '14 at 20:59
  • Again, symetry tells us $\mathrm{E}[X_1\mid \bar{X}_n] = \dots = \mathrm{E}[X_n\mid \bar{X}_n]$. Hence, $\mathrm{E}[X_1\mid \bar{X}_n]=(\mathrm{E}[X_1\mid \bar{X}_n]+\dots+\mathrm{E}[X_n\mid \bar{X}_n])/n=\mathrm{E}[\bar{X}_n\mid \bar{X}_n]=\bar{X}_n$. – Zen Jan 28 '14 at 23:37
  • Symetry here means two things: 1) IID $X_i$'s; 2) The symetry (with respect to the $X_i$'s) of the conditioning sigma-field (generated by $\bar{X}_n$). – Zen Jan 28 '14 at 23:41
  • By the way, it's a good exercise to make the symetry argument rigorous. – Zen Jan 28 '14 at 23:42
2

The OP has apparently found the way, so I am posting an answer.

I will denote $Z \equiv \sum_{i=1}^n X_i$. By linearity of the expected value we have $$E \left( X_1+X_2+3X_3 |Z \right)= E \left( X_1 |Z \right)+E \left( X_2 |Z \right)+3E \left(X_3 |Z \right)$$

Since the variables are i.i.d. they are also exchangeable,at least with respect to $Z$ (to which they have a symmetric relationship), so the three conditional expected values will be equal:

$$E \left( X_1+X_2+3X_3 |Z \right)= 5E \left( X_1 |Z \right)$$

Moreover, it is a known result that the conditional distribution of $X_1$ conditional on $Z=k$ is a Binomial,

$$X_1 | Z=k \sim Bin\left(k, \frac {E(X_1)}{E(Z)}\right) = Bin\left(k, 1/n\right)$$

and so

$$5E \left( X_1 |Z=k \right) = 5\frac kn$$

Our conditional expectation is viewed as a function of $Z$, is not conditioned just on $Z$ acquiring a specific value. Generalizing the last equation we obtain

$$5E \left( X_1 |Z \right) = \frac 5n Z= 5 \frac 1n \sum_{i=1}^n X_i$$

Note that

$$E \left( X_1 |Z \right) \rightarrow_p E(X_1) \;\;\text {as}\;\; n\rightarrow \infty$$

which should be intuitive.

Alecos Papadopoulos
  • 52,923
  • 5
  • 131
  • 241
  • You might consider revising slightly the sentence beginning with "*Since the variables are i.i.d....*". The fact that they are i.i.d. does not imply that they are exchangeable *conditional* on an *arbitrary* $Z$. This $Z$ is special! – cardinal Jan 19 '14 at 01:38
  • @cardinal I thought it self-understood but I will point it out, sure. Thanks. – Alecos Papadopoulos Jan 19 '14 at 01:43
  • Thank you very much. It's helpful to have two ways to arrive at the result. – JohnK Jan 19 '14 at 10:04