1

Are there any specific families of probability distribution which are not exchangeable by construction?

I was thinking that the Hyper Geometric distribution would not be since it models random sampling without replacement, but I'm not sure?

Sean Easter
  • 8,359
  • 2
  • 29
  • 58
Bex
  • 13
  • 3
  • 2
    In its usual sense, "exchangeable" applies only to multivariate distributions. Evidently you are using this term differently. Please, then, explain what you understand "exchangeable" to mean. – whuber Aug 12 '16 at 13:59

2 Answers2

9

Exchangeability doesn't apply to a distribution, but a sequence of random variables. From wikipedia:

Formally, an exchangeable sequence of random variables is a finite or infinite sequence $X_1, X_2, X_3, ...$ of random variables such that for any finite permutation $\sigma$ of the indices $1, 2, 3, ...$, [...] the joint probability distribution of the permuted sequence $X_{\sigma (1)},X_{\sigma (2)},X_{\sigma (3)},\dots$ is the same as the joint probability distribution of the original sequence.

A sequence $\textbf{k}$ of draws from a hypergeometric distribution has mass $P(\textbf{k}) = \prod_{i=0}^m P(k_i)$, which you can see is invariant to permutation, and thus exchangeable.

What I think you might be curious about is whether the sequence of successes summarized by a hypergeometric is exchangeable. It is: The PMF depends only on $k$, the count of successes, and not their order.

For examples of sequences that are non-exchangeable, any sequence $X_1, X_2, X_3, ...$ in which each $X_i$ depends solely on the prior random variable $X_{i-1}$ will do. (Meaning, $X_i \perp\mspace{-8mu}\perp X_{j \lt i-1} | X_{i-1}$)

Say, $X_i|x_{i-1} \sim N(x_{i-1},1)$, with $X_1 \sim N(0,1)$:

import numpy as np
from scipy.stats import norm

X = np.array([0,1,2,3])
X_permuted = np.random.permutation(X)
X_diff = np.diff(X)
X_permuted_diff = np.diff(X_permuted)

assert norm.pdf(X_diff).prod() == norm.pdf(X_diff_perm).prod()

You can also view this example as $X_i = x_{i-1} + \epsilon_i$ with $\epsilon_i \sim N(0,1)$, which provides a—helpful, I hope—counterexample: If you take the differences first, they're exchangeable.

# This won't throw an error
X_diff_permuted = np.random.permutation(X_diff)
assert norm.pdf(X_diff).prod() == norm.pdf(X_diff_permuted).prod()
Sean Easter
  • 8,359
  • 2
  • 29
  • 58
  • Seems correct, if the trivial possibility of the iid case is excluded. Note that this is the Markov property. (I'll delete my previous comments as they are now obsolete) – Juho Kokkala Aug 12 '16 at 18:47
  • 2
    @JuhoKokkala Yes, "the Markov property" was what I intended and would have been much more succinct of me, but I didn't want to send the OP toward more technical terminology. Thanks again! – Sean Easter Aug 12 '16 at 19:25
1

I am not sure I completely understand your question. The term exchangeability refers to a sequence of random variables. A sequence of random variables is exchangeable when is invariant to random permutations. A sequence of random variable is exchangeable as long as the random variables are i.i.d.. Therefore if you have lets say some samples for any distribution that are independent one another then you have an exchangeable sequence of random variables.

Does this answer your question?

Akis
  • 21
  • 4
  • To build on Akis answer, the hypergeometric distribution is, indeed, exchangeable. If we use the classical example of hypergeometric distributions, where we are randomly sampling white and black marbles from an urn without replacement: the probability of any particular sequence of white and black marbles depends only on the number of black and white marbles, not on the order in which they appear. The probability of drawing, say, a white marble on the ith draw is just K/N, where K is the number of white marbles in the urn (where N is the total number of marbles). – Ryan Simmons Aug 12 '16 at 13:27
  • 1
    @Ryan Your comment appears to confuse *distributions* with *sampling procedures.* As noted elsewhere in this thread, "exchangeable" does not apply to a distribution or distribution family (except perhaps in a mathematically trivial sense). – whuber Aug 12 '16 at 14:00
  • 2
    @whuber. You are correct, I typed my comment quickly and was imprecise with my language. I was simply attempting to build on Akis answer to specifically refer back to the OP's mention of the hypergeometric distribution, but you are right I should have referred to a sequence of draws from that distribution rather than the distribution itself. Luckily, Sean Easter seems to have provided the same information in a much more accurate manner, so hopefully my lackadaisical comment won't confuse anyone! – Ryan Simmons Aug 12 '16 at 14:13