2

As we know the expectation of a RV $X$ or a function, say $g(X)$, of $X$, both with pdf $p_{X}(x)$ is

$$ \begin{array}{*{20}{c}} {X \sim {p_X}(x):}&{E[X] = \int {x.{p_X}(x)dx} }\\ {g(X) \sim {p_X}(x):}&{E[X] = \int {g(x).{p_X}(x)dx} } \end{array} $$

So my question is, can we say that When $X$ is just an argument the mean of pdf $p(x)$ is the same as the expected value of $X$, i.e. $E[X]$; but for a function of this RV ,say $g(X)$, it is not true.

Also as mentioned in this topic:

"The expectation is the average value or mean of a random variable not a probability distribution."

So in general: The expected value of a RV is not always the same as the mean of the corresponding pdf. Is my interpretation correct?

StubbornAtom
  • 8,662
  • 1
  • 21
  • 67
sci9
  • 337
  • 3
  • 13
  • There is no difference between expectation of $X$ and expectation of the distribution of $X$. – StubbornAtom Jun 02 '19 at 10:27
  • Thank you for comment. And about the expectation of a function of $X$ and expectation of the distribution of $X$? @StubbornAtom – sci9 Jun 02 '19 at 10:29
  • 1
    If $p$ is the pdf of $X$, then mean of $X$=$\int xp(x)\,dx$. And for a function $g$, mean of $g(X)$ is $\int g(x)p(x)\,dx$, which follows from [this](https://en.m.wikipedia.org/wiki/Law_of_the_unconscious_statistician) theorem. – StubbornAtom Jun 02 '19 at 10:37
  • @StubbornAtom Thank you for the link. So we say that when the distribution of $g(X)$, i.e. $p_{g(X)}(x)$, is known, $E(g(X))$ is the same as mean of pdf $p_{g(X)}(x)$? – sci9 Jun 02 '19 at 10:53
  • 1
    Yes, $E(g(X))=\int x p_{g(X)}(x)\,dx$. – StubbornAtom Jun 02 '19 at 11:01
  • https://en.wikipedia.org/wiki/Law_of_the_unconscious_statistician – whuber Jun 02 '19 at 15:48

1 Answers1

2

The expectation of a (Borel measurable) function $g$ relative to a probability density $f(x)$ is defined to be the integral of $g(x)f(x)\mathrm{d}x$ provided the integral of $|g(x)|f(x)\mathrm{d}x$ is finite. This is always equal to the expectation of $g(X)$ whenever $X$ is a random variable with $f$ for its density.

This is a fundamental result of probability theory and so is well worth learning and understanding. Although it has frequently been quoted on these pages, I don't believe it has been rigorously stated here, nor has any sketch of its proof been shown. For those details, read on.


Let's get clear about the definitions.

A random variable $X$ associates numerical values to outcomes in a probability space $(\Omega, \mathfrak F, \mathbb P).$ Its expectation is the mathematical expression of the average of $X$ as weighted by the probability, written

$$E[X] = \int_\Omega X(\omega)\,\mathrm{d}\mathbb{P}(\omega).$$

In this expression, which is a Lebesgue integral, "$\omega$" refers to elements of the sample space $\Omega$, $X(\omega)$ is the value associated by $X$ to $\omega,$ and $\mathrm{d}\mathbb{P}(\omega)$ can be understood as its proper weight in this average.

Similarly, when $g$ is a function of the possible values of $X$ (so it assigns numbers to numbers) and $g(X)$ also is a random variable, this formula shows $g(X)$ has an expectation

$$E[g(X)] = \int_\Omega g(X(\omega))\,\mathrm{d}\mathbb{P}(\omega).$$

The (probability) distribution $F_X$ of a random variable $X$ is a probability function defined on certain "nice" sets of numbers, the Borel sets. For any number $x,$ it is determined by the rule

$$F_X(x) = \mathbb{P}\left(\left\{\omega\in\Omega\mid X(\omega)\le x\right\}\right).$$

In words: the value of the distribution function $F_X$ at the number $x$ is the chance that $X$ will not exceed $x.$

When $F_X$ has a derivative $f_X,$ the Fundamental Theorem of Calculus says $F_X$ can be recovered by integrating $f_X:$

$$F_X(x) = \int_{-\infty}^x f_X(x)\mathrm{d}x.$$

In this case, we say $X$ has a probability density function (pdf) $f_X.$ Such a density function can be considered an assignment of a non-negative number $f_X(x)$--the "probability density at $x$"--to every number $x.$ This makes it a different kind of mathematical object than $X.$ Nevertheless, the two objects enjoy a fundamental relationship.

The Law of the Unconscious Statistician assserts the expectation of any sufficiently nice (i.e., measurable) function $g$ applied to $X,$ written above as an abstract integral over $\Omega,$ can always be computed as an integral over $f_X$ when $X$ has a pdf:

LOTUS (the Law of the Unconscious Statistician): $$E[|g(X)|] = \int_{-\infty}^\infty |g(x)| f(x)\mathrm{d}x$$ and, when this quantity is finite, $$E[g(X)] = \int_{-\infty}^\infty g(x) f(x)\mathrm{d}x.$$

This is not straightforward to prove. The standard demonstration echoes the definition of the Lebesgue integral: basically, you have to start from scratch by defining the two kinds of integrals over the simplest possible functions (those that take on only the values $0$ and $1$) and gradually generalizing them to more complicated functions, checking at each step that LOTUS holds. The stages of generality of $g$ are:

  1. Indicator functions (measurable functions with values in $0$ and $1$).

  2. Finite sums of indicator functions multiplied by positive constants ("simple functions").

  3. Non-negative (Borel) measurable functions. These can be approximated by simple functions.

  4. General measurable functions. These can be expressed as differences of non-negative measurable functions.

For details, see the reference.

Reference

Steven Shreve, Stochastic Calculus for Finance II: Continuous-Time Models (Springer 2000), section 1.5.

whuber
  • 281,159
  • 54
  • 637
  • 1,101