Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [conditional-expectation]

A conditional expectation is the expectation of a random variable, given information on another variable or variables (mostly, by specifying their value).

A conditional expectation, or conditional mean, is the expectation of a random variable, given information on another variable or variables (mostly, by specifying their value). [The expected height of an adult US male is different from the expected height of an adult US male, born in 1942 to Micronesian parents.

For discrete variables, $\text{E}(X|Y=y) = \sum x \ \text{P}(X=x|Y=y)$, while for continuous variables, $\text{E}(X|Y=y)= \int x \ f_X(x|Y=y) \ dx$, where the sum and integral are over the possible values taken by $x$.

An expectation conditioned on some subset of the possible values taken by $Y$, such as $\text{E}(X\mid a < Y < b )$, say, is also a conditional expectation.

For an in-depth treatment of conditional expectations see:
Billingsley, P. (1995) "Probability and Measure", 3rd edition, John Wiley & Sons Inc., New York

747 questions
92
votes
2 answers

Subscript notation in expectations

What is the exact meaning of the subscript notation $\mathbb{E}_X[f(X)]$ in conditional expectations in the framework of measure theory ? These subscripts do not appear in the definition of conditional expectation, but we may see for example in this…
Emile
  • 3,150
  • 2
  • 20
  • 17
62
votes
3 answers

A generalization of the Law of Iterated Expectations

I recently came across this identity: $$E \left[ E \left(Y|X,Z \right) |X \right] =E \left[Y | X \right]$$ I am of course familiar with the simpler version of that rule, namely that $E \left[ E \left(Y|X \right) \right]=E \left(Y\right) $ but I was…
JohnK
  • 18,298
  • 10
  • 60
  • 103
31
votes
4 answers

Intuition for Conditional Expectation of $\sigma$-algebra

Let $(\Omega,\mathscr{F},\mu)$ be a probability space, given a random variable $\xi:\Omega \to \mathbb{R}$ and a $\sigma$-algebra $\mathscr{G}\subseteq \mathscr{F}$ we can construct a new random variable $E[\xi|\mathscr{G}]$, which is the…
Nicolas Bourbaki
  • 1,080
  • 9
  • 22
27
votes
4 answers

Problem with proof of Conditional expectation as best predictor

I have an issue with the proof of $E(Y|X) \in \arg \min_{g(X)} E\Big[\big(Y - g(X)\big)^2\Big]$ which very likely reveal a deeper misunderstanding of expectations and conditional expectations. The proof I know goes as follows ( another version…
Martin Van der Linden
  • 739
  • 1
  • 6
  • 13
26
votes
4 answers

What is the difference between $E(X|Y)$ and $E(X|Y=y)$?

Generally, What is difference between $E(X|Y)$ and $E(X|Y=y)$? Former is function of $y$ and latter is function of $x$? It's so confusing..
신범준
  • 371
  • 1
  • 3
  • 5
19
votes
2 answers

How should I mentally deal with Borel's paradox?

I feel a bit uneasy with how I've mentally dealt with Borel's paradox and other associated "paradoxes" dealing with conditional probability. For those who are reading this that aren't familiar with it, see this link. My mental response up to this…
guy
  • 7,737
  • 1
  • 26
  • 50
18
votes
2 answers

Law of total variance as Pythagorean theorem

Assume $X$ and $Y$ have finite second moment. In the Hilbert space of random variables with second finite moment (with inner product of $T_1,T_2$ defined by $E(T_1T_2)$, $||T||^2=E(T^2)$), we may interpret $E(Y|X)$ as the projection of $Y$ onto the…
renrenthehamster
  • 347
  • 3
  • 7
18
votes
2 answers

Conditional expectation of exponential random variable

For a random variable $X\sim \text{Exp}(\lambda)$ ($\mathbb{E}[X] = \frac{1}{\lambda}$) I feel intuitively that $\mathbb{E}[X|X > x]$ should equal $x + \mathbb{E}[X]$ since by the memoryless property the distribution of $X|X > x$ is the same as that…
mchen
  • 710
  • 1
  • 6
  • 15
17
votes
3 answers

Why do we care more about test error than expected test error in Machine Learning?

In Section 7.2 of Hastie, Tibshirani, and Friedman (2013) The Elements of Statistic Learning, we have the target variable $Y$, and a prediction model $\hat{f}(X)$ that has been estimated from a training set $\mathcal{T} = \{Y_1, ..., Y_N, X_1, ...,…
Colin T Bowers
  • 745
  • 6
  • 23
16
votes
4 answers

Expected value of sample median given the sample mean

Let $Y$ denote the median and let $\bar{X}$ denote the mean, of a random sample of size $n=2k+1$ from a distribution that is $N(\mu,\sigma^2)$. How can I compute $E(Y|\bar{X}=\bar{x})$? Intuitively, because of the normality assumption, it makes…
JohnK
  • 18,298
  • 10
  • 60
  • 103
15
votes
4 answers

If $X_1,\cdots,X_n \sim \mathcal{N}(\mu, 1)$ are IID, then compute $\mathbb{E}\left( X_1 \mid T \right)$, where $T = \sum_i X_i$

Question If $X_1,\cdots,X_n \sim \mathcal{N}(\mu, 1)$ are IID, then compute $\mathbb{E}\left( X_1 \mid T \right)$, where $T = \sum_i X_i$. Attempt: Please check if the below is correct. Let say, we take the sum of the those conditional…
learning
  • 283
  • 1
  • 6
14
votes
2 answers

OLS as approximation for non-linear function

Assume a non-linear regression model \begin{align} \mathbb E[y \lvert x] &= m(x,\theta) \\ y &= m(x,\theta) + \varepsilon, \end{align} with $\varepsilon := y - m(x,\theta)$. I heard someone say that OLS always estimates…
Jesper for President
  • 5,049
  • 1
  • 18
  • 41
12
votes
0 answers

Conditional expectation subscript notation

This should be a relatively simple question. I'm trying to confirm my understanding of the subscript notation on expectations when the subscript denotes a conditioning. In the example $$E_{Y|X}[(Y-f(X))^2|X]$$ the subscript denotes the…
tjnel
  • 852
  • 10
  • 14
12
votes
3 answers

Variance of sample mean of bootstrap sample

Let $X_{1},...,X_{n}$be distinct observations (no ties). Let $X_{1}^{*},...,X_{n}^{*}$denote a bootstrap sample (a sample from the empirical CDF) and let $\bar{X}_{n}^{*}=\frac{1}{n}\sum_{i=1}^{n}X_{i}^{*}$. Find $E(\bar{X}_{n}^{*})$ and…
rrruss
  • 121
  • 1
  • 5
12
votes
2 answers

How to compute conditional expectations with respect to a sigma field?

Example: Toss a coin twice. Letting $\mathbb P$ be a probability measure, suppose $\mathbb P(HH)=p^2,\mathbb P(HT)=\mathbb P(TH)=p(1-p), \mathbb P(TT)=(1-p)^2.$ I would like to answer the following questions: Define $Y$ to be the number of heads…
pual ambagher
  • 739
  • 5
  • 13
1
2 3
…
49 50