4

If the random variables $X_1,...,X_n$ are I.I.d. when conditioned on an unknown parameter $\theta$, where $p(\theta)$ represents our prior beliefs about $\theta$, it follows from the commutativity of the product that $X_1,...,X_n$ are exchangeable.

In my professor's lecture notes, he goes on to write that because $X_1,...X_n$ are exchangeable, it holds that $X_1,...X_n$ are dependent. I'm having a hard time understanding why this is the case.

What if I rolled a dice three times? Then the outcomes are exchangeable, I.e. $P[(4,5,6)]=P[(5,4,6)]=P[(6,4,5)]$, yet the three rolls are independent. Am I missing something?

DavidSilverberg
  • 739
  • 6
  • 18

3 Answers3

6

The phrase is true because if they weren't dependent, you can't learn about unsampled values. That is, learning $X_1,...,X_n$ does not improve your prediction for $X_{n+1}$ no matter how large $n$ is. Mathematically "independence" is saying

$$p(X_{n+1}=x|X_1,...,X_n,I)=p(X_{n+1}=x|I)$$

In your example the dice rolls are not independent, provided the prior is not a discrete mass at a single value. To prove this in the standard case, take probability that a six comes up, using a uniform prior. The above probability is equal to the beta binomial distribution with expected value $$E(X_{n+1}|X_1,...,X_n,I)=\frac{y+1}{n+2}$$ where $y=X_1+...+X_n$ This clearly depends on $X_1,...,X_n$, for all $n>1$ meaning the exchangeable data are dependent.

Another way to think is "independent" means that "posterior predictive" = "prior predictive"

probabilityislogic
  • 22,555
  • 4
  • 76
  • 97
2

The statement:

"X1,...Xn are exchangeable $\Rightarrow$ that X1,...Xn are dependent"

is false in general case and the example you give in your question gives a correct counter-example.

Maybe, your professor wanted to outline that the converse of :

" X1,...Xn are independent $\Rightarrow$ X1,...Xn are exchangeable "

is false (which is different from the above proposition). For an example of counterexample for this last statement, see the example developed in another answer : https://stats.stackexchange.com/a/3523./14346

peuhp
  • 4,622
  • 20
  • 38
2

Repeated dice rolls are not independent by default

Three dice rolls are independent if and only if you have certainty about their probabilities before making the rolls. For example, if it's certain that you're rolling a single fair die with P(1)=P(2)=P(3)=P(4)=P(5)=P(6)=1/6, then the dice rolls are independent.

However, in the general case, if you don't have certainty about the probabilities, they are dependent events conditioned on a hidden variable i.e. the nature of that die. You'd expect the results to be correlated - if the first 99 rolls show a six 50 times, then that's probably not a fair die and the probability of rolling a 6 in the 100th roll is much larger than 1/6.

Peteris
  • 363
  • 1
  • 5