Conditional expectation of random variables defined off of each other

Question

First of all, when we say that $X_n \sim \text{Unif}(0,X_{n-1})$, what does that mean, rigorously? Does it mean that for every $\omega \in \Omega$, $X_n(\omega)\sim \text{Unif}(0,X_{n-1}(\omega))$? This doesn't make much sense as $X_n(\omega)$ is a number and not a random variable. Or does it mean that for every $\omega \in \Omega$ there is some auxiliary $X'_n \sim \text{Unif}(0,X_{n-1}(\omega))$, and then $X_n(\omega) = X'_n(\omega)$? Once we have that cleared up, I have another question:

Let's say that we have $X_n \sim \text{Unif}(0,X_{n-1})$ where $X_0=1$. I've been told that defining $\mathcal A_n = \sigma[X_0, \ldots, X_{n}]$, $$\mathbb E(X_{n+1}|\mathcal A_n) = \mathbb E(X_{n+1}|X_n) = \mathbb E[\text{Unif}(0,X_{n})] = \frac{0+X_{n}}{2}$$ Similarly, for $Z_{n+1} \sim \text{Poisson}(Z_n)$ and $\mathcal A_n$ defined similarly as in the above case, I've been told that $$\mathbb E(Z_{n+1}|\mathcal A_n) = \mathbb E[\text{Poisson}(Z_{n})] = Z_n$$ and also for $Y_{n+1} \sim2 \text{Binom}(Y_n, p)$ $$\mathbb E(Y_{n+1}|\mathcal A_n) = 2\mathbb E[\text{Binom}(Y_{n}, p)] = 2Y_n p$$

Question: What are the proofs of these three statements (using the modern probability theory approach and conditional expectation properties such as those listed here)? Is this sort of pattern generally true for random variables defined off of each other like this?

The only formula from conditional expectations that I think would apply here is that $\mathbb E(Y|\mathcal D) =_\text{a.s.} \mathbb E[Y]$ if $\mathcal F(Y)$ and $\cal D$ are independent. However, in the above case I don't think the sigma fields are independent, so I don't know how to go about proving the nice equations above.

Answer 1

Your first question is key, so let's focus on it. You are concerned about a bivariate random variable $(X_{n-1},X_n)$ with a probability distribution somehow defined by giving $X_{n-1}$ a distribution and then defining the distribution function of $X_n$ in terms of the random variable $X_{n-1}.$

There are many subtleties involved, so I write the following in the hope that exploring the consequences of the definitions in detail and carrying out the calculations explicitly will reveal what actually is being done when the distribution of one random variable is defined in terms of another random variable. We will see that this makes sense, but by means of the first example in the question (the uniform distributions) we will see that it does not uniquely determine the random variables themselves.

---

You are working with conditional expectations, so let's begin by expressing probabilities in terms of expectations. Let $x\in\mathbb{R}$ be a value at which we wish to compute the distribution function of $X_n,$

$$F_n(x) = \Pr(X_n \le x) = \phi(x;X_{n-1})$$

where $\phi$ defines the distribution of $X_n$ in terms of $X_{n-1}.$ For instance, assuming $\Pr(X_n\gt 0) = 1$ for simplicity and taking $x\gt 0,$ $\phi$ might be a uniform distribution function

$$\phi(x;X_{n-1}) = \frac{\min(X_{n-1},x)}{X_{n-1}} = \min\left(1, \frac{x}{X_{n-1}}\right) .$$

The probability determined by $F_n$ can be expressed in terms of the indicator function $\mathcal{I}(X_n\le x)$ as

$$\Pr(X_n \le x) = E\left[\mathcal{I}(X_n\le x)\right].$$

Because a property of $X_n$ has been expressed in terms of $X_{n-1},$ to be explicit we must consider this to be a conditional expectation,

$$E\left[I(X_n\le x)\right] = E\left[\mathcal{I}(X_n\le x)\mid X_{n-1}\right].$$

We may assemble all the foregoing into the formula

$$ E\left[ \mathcal{I}(X_n\le x)\mid X_{n-1}\right] = \phi(x;X_{n-1}).\tag{*}$$

Does this make sense? Let's check it against a definition of conditional expectation. $X_n$ is a random variable with respect to some sigma algebra $\mathfrak{F}_n$ defined on a probability space $(\Omega, \mathfrak{F}_n, \mathbb{P}).$ Let $\mathfrak{F}_{n-1}\subset \mathfrak{F}_n$ be the sigma algebra generated by the conditioning variable $X_{n-1}.$ Then the conditional expectation in $(*)$ is a $\mathfrak{F}_{n-1}$-measurable random variable, allowing us to write expressions like

$$E\left[ \mathcal{I}(X_n\le x)\mid X_{n-1}\right](\omega)$$

to refer to its value on an outcome $\omega\in\Omega.$ Notice that the right hand side of $(*)$ is also $\mathfrak{F}_{n-1}$-measurable provided $y \to \phi(x;y)$ is a measurable function for every $x,$ because it's a function of $X_{n-1}.$ So far so good: at least $(*)$ is equating two comparable mathematical objects!

The defining property of a conditional expectation is

$$\int_A E\left[ \mathcal{I}(X_n\le x)\mid X_{n-1}\right](\omega)\,\mathrm{d}\mathbb{P}(\omega) = \int_A \mathcal{I}(X_n(\omega)\le x)\, \mathrm{d}\mathbb{P}(\omega)\tag{**}$$

for all $\mathfrak{F}_{n-1}$-measurable sets $A.$

Substituting $(*)$ gives

$$\int_A \phi(x;X_{n-1}(\omega))\,\mathrm{d}\mathbb{P}(\omega) = \int_A \mathcal{I}(X_n(\omega)\le x)\, \mathrm{d}\mathbb{P}(\omega)$$

for all events $A\in\mathfrak{F}_{n-1}.$ Thus, specifying $\phi$ amounts to specifying all possible values of the integral on the right for all $X_{n-1}$-measurable sets $A.$ That's enough to determine $X_n$ up to distribution because if $Y$ is another variable with this property, then for all $A\in\mathfrak{F}_{n-1},$

$$0 = \int_A ( \mathcal{I}(X_n(\omega)\le x) - \mathcal{I}(Y(\omega)\le x))\, \mathrm{d}\mathbb{P}(\omega)$$

implies the two indicator functions are almost surely ($\mathfrak{F}_{n-1}$) equal. In particular, setting $A=\Omega$ gives

$$0 = \Pr(X_n \le x) - \Pr(Y \le x),$$

showing that $X_n$ and $Y$ are identically distributed.

---

A counterexample might help reinforce the fact that the random variable $X_n$ is usually not uniquely determined by $\phi$ and $X_{n-1}.$ Let $\Omega = [0,1]\times [0,1]$ with the usual Borel sigma algebra and Lebesgue measure $\mathbb P.$ I will exploit the obvious fact that the map

$$\iota: \Omega\to\Omega;\quad \iota(\omega_1,\omega_2) = (\omega_1,1-\omega_2)$$

(which merely flips the square $\Omega$ upside down) preserves all the essential properties of this probability space.

The random variable defined by

$$X_{n-1}(\omega_1,\omega_2) = \omega_1$$

has a Uniform$(0,1)$ distribution. The random variables

$$X_n(\omega_1,\omega_2) = \omega_1\omega_2$$

and

$$X_n^{(2)}(\omega_1,\omega_2) = \omega_1(1-\omega_2)$$

are identically distributed because they are related by $\iota.$ To compute their common conditional distribution, note that the sigma algebra $\sigma(X_{n-1}) = \mathfrak{F}_{n-1}$ is generated by the sets of the form

$$\Omega_{t} = \{(\omega_1,\omega_2)\mid \omega_1\le t\}.$$

Visualize these as slicing the square $[0,1]\times[0,1]$ vertically at the location $t;$ the two halves are measurable in the subalgebra and all $\mathfrak{F}_{n-1}$-measurable sets consist of collections built out of such slices.

It suffices therefore to let $A = \Omega_{t}$ for arbitrary $t$ and compute

$$\eqalign{\int_A \mathcal{I}(X_n(\omega)\le x)\, \mathrm{d}\mathbb{P}(\omega) &= \int_0^1\int_0^t \mathcal{I}(\omega_1\omega_2 \le x)\, \mathrm{d}\omega_1\,\mathrm{d}\omega_2\\ &= \int_0^1\int_0^t \min\left(1,\frac{x}{\omega_1}\right)\, \mathrm{d}\omega_1\,\mathrm{d}\omega_2\\ &= \int_0^1\int_0^t \phi(x, \omega_1)\, \mathrm{d}\omega_1\,\mathrm{d}\omega_2\\ &= \int_0^1\int_0^t \phi(x, X_n(\omega))\, \mathrm{d}\omega_1\,\mathrm{d}\omega_2\\ &= \int_A \phi(x, X_n(\omega))\, \mathrm{d}\mathbb{P}. }$$

Everything in this chain of equalities merely substitutes a definition or previous equality except for the first and last lines, which apply Fubini's Theorem, and the move from the first line to the second, which is simple arithmetic.

Thus, the integrand at the end must be the conditional expectation because it has been seen to satisfy the defining property $(**):$

$$E\left[ \mathcal{I}(X_n\le x)\mid X_{n-1}\right] = \phi(x, X_n(\omega)).$$

Although this equation is true of both $X_{n}$ and $X_{n}^{(2)},$ these are distinct random variables. Indeed,

$$X_{n}(\omega_1,\omega_2) - X_{n}^{(2)}(\omega_1,\omega_2) = \omega_1\omega_2 - \omega_1(1-\omega_2) = \omega_1(2\omega_2 - 1)$$

is almost surely nonzero.

Answer 2

Definitions

Definition 1: If $\mathcal F \subseteq \mathcal G$ are two $\sigma$-fields, and $X$ a $\mathcal G$-measurable integrable random variable, then $\mathbb E[X | \mathcal F]$ is defined as any $\mathcal F$-measurable random variable $Y$, such that $\mathbb E[Y;A]=\mathbb E[X;A]$ for every $A \in \mathcal F$. Here $\mathbb E[X;A]$ is a notation for $\int_AX\,d\mathbb P$.

Definition 2: We define conditional probability as $\mathbb P(A | \mathcal F)= \mathbb E[1_A|\mathcal F]$.

https://math.stackexchange.com/questions/2373097/condition-on-sigma-algebra/2373105#2373105

$X_n\sim Unif(0,X_{n-1})$ it means $X_n|X_{n-1}\sim Unif(0,X_{n-1})$

it also means $X_n|X_{n-1}=t\sim Unif(0,t)$

rigorously it is a conditional probability. How it define? It define bayed on conditional Expectation , http://www2.stat.duke.edu/courses/Fall17/sta711/lec/wk-10.pdf ,

on the Other hand

$$P(X_n\leq a|X_{n-1})=E(1_{X_n\leq a}|X_{n-1})=E(1_{X_n\leq a}|\sigma(X_{n-1}))$$

This type of conditional probability is unified definition that other definitions( continues variable, discrete variables, events,mixture variables) are a special case of this. (like a patch for old definition, $P(A|B)=\frac{P(AB)}{P(B)}$, problem with continues variable, like in this situation,$B=\{X_{n-1}=t\}$ so $P(B)=0$).

we saw this (define a variable conditioned another variable) before in Bayesian approach,

$$X|\mu \sim N(\mu , 1)$$, that conditioned based a variable $\mu\sim N(0,1)$,

and in Metropolis-Hastings Method that conditioned based on previous observation.

"Does it mean that for every $\omega \in \Omega$ $X_n(\omega)∼Unif(0,X_{n−1}(\omega))$?"

Only enough for almost sure of them.

$$E(X_{n+1}|A_n) = E(X_{n+1}|\sigma(X_1,\cdots ,X_n)) = E(X_{n+1}|X_1,\cdots ,X_n)= E(X_{n+1}|X_n)= \frac{X_n}{2}$$

all other question solve similar.

Answer 3

So first: When you define a random variable $X_{n+1}$ to depend on the value of a different random variable $X_n$ you are effectively defining the conditional probability $P(X_{n+1}|X_n)$ as long as $P(X_n)$ was well defined this defines a probability distribution $P(X_{n+1}, X_n)$ over the two variables with the corresponding outer product $\Omega$ and $\sigma$-algebra. I think this is what you rigorously define when you write a distribution for a new random variable depending on an existing one.

For your second question I would first comment on the writing: Generally, I would expect you to condition on the other random variables directly, not on the $\sigma$-algebra for them. If you mean something else for those, please clarify.

Then the general part for your examples is that if you define $X_{n+1}$ only based on $X_n$ you can ignore all previous random variables for calculating the conditional expectation. This is indeed true. There are many ways of showing this. Maybe the simplest is to write out that $P(X_0,\dots,X_n)=P(X_0)\prod_{i=1}^n P(X_i|X_{i-1})$, and observing that $P(X_{n+1}| X_n)$ is indeed the conditional probability as we would assume based on the naming. Then the raw definition of the conditional expectation $\mathbb{E}(X_{n+1}|X_0,\dots,X_n)$ is:

\begin{eqnarray} \mathbb{E}(X_{n+1}|X_0,\dots,X_n) &=& \int x_{n+1} dP(X_{n+1}|X_0,\dots,X_n)\\ &=& \int x_{n+1} d\left[\frac{P(X_{n+1},X_0,\dots,X_n)}{P(X_0,\dots,X_n)}\right]\\ &=& \int x_{n+1} dP(X_{n+1}|X_n) \end{eqnarray}

as all other P cancel in the ratio.

Note that this expectation is not the expectation for the (n+1)th variable $\mathbb{E}(X_{n+1})$, which is equal to the expectation of the marginal $P(X_{n+1})$, which will not necessarily have a simple distribution. If you are interested in this expectation, you'll have to find a way to calculate $\mathbb{E}(X_{n+1})$ from $\mathbb{E}(X_{n})$ and do the induction.