8

This question came in the context of understanding how to get a distribution of a sum of two iid random variables. I'm working through the top answer to this question Consider the sum of $n$ uniform distributions on $[0,1]$, or $Z_n$. Why does the cusp in the PDF of $Z_n$ disappear for $n \geq 3$?, trying to understand why characteristic functions behave the way they do.

The title is the whole question.

Anton
  • 443
  • 2
  • 10

2 Answers2

11

Let's suppose we could find a (measurable) function $\chi$ defined on the real numbers with the property that

$$\chi(a + b) = \chi(a)\chi(b)$$

for all numbers $a$ and $b$ and for which there is a finite positive number $M$ for which $|\chi(a)| \le M$ for all $a$. Notice how $\chi$ relates addition (which is the fundamental operation appearing in a convolution) and multiplication.

Why are these properties useful? Suppose $X$ and $Y$ are independent random variables. Let $t$ be any real number. Then (taking up these two properties in reverse order)

  1. $\mathbb{E}(\chi(tX)) \le \mathbb{E}(|\chi(tX)|) = \mathbb{E}(M) = M \lt \infty$ (with a similar expression for $Y$) shows that the expectations of the random variables $\chi(tX)$ and $\chi(tY)$ exist and are finite, with a uniform bound $M$ independent of $t$.

    This procedure of taking a random variable $X$ and transforming it into the function

    $$t \to \mathbb{E}(\chi(tX)) = (\operatorname{cf}_\chi(X))(t)$$

    thereby assigns a well-defined, bounded function $\operatorname{cf}_\chi(X)$ to every random variable $X$--no matter what awful properties $X$ might have.

  2. $\mathbb{E}(\chi(t(X+Y))) = \mathbb{E}(\chi(tX)\chi(tY)) = \mathbb{E}(\chi(tX))\mathbb{E}(\chi(tY))$ because $X$ and $Y$ are independent. Written slightly differently,

    $$(\operatorname{cf}_\chi(X+Y))(t) = ((\operatorname{cf}_\chi(X))(\operatorname{cf}_\chi(Y)))(t)$$

    That is, the transformation $\operatorname{cf}_\chi$ converts convolution (addition of random variables) into (pointwise) multiplication of functions.

Much more can be said: see the literature on Fourier Analysis. But in the meantime, the question has been answered in a way that shows "time" and "frequency" may be red herrings: this fundamental property of converting convolution into multiplication relies only on the existence of a nice $\chi$.

The only real-valued functions with the defining properties of $\chi$ are $\chi(a) = 1$ and $\chi(a) = 0$. They lead to nothing useful. But if we allow $\chi$ to have complex values, then $\chi(a) = \exp(ia)$ is one such function and it produces useful results. (Moreover, all such $\chi$ are derived from this one: they must be of the form $a\to \exp(ia\lambda)$ for some fixed real number $\lambda$.) In this case $\operatorname{cf}_\chi(X)$ is called the characteristic function of $X$.


It's not hard to see that when $\chi$ is not identically zero, $|\chi(a)|$ must always equal $1$, no matter what $a$ is. Such functions are called (complex) multiplicative characters (of the additive group of real numbers).

whuber
  • 281,159
  • 54
  • 637
  • 1,101
2

The answer depends on what you're looking for in the answer.

For instance, to me this was the answer, and the rest was just the details: $$e^a\cdot e^b=e^{a+b}$$ It's all about the exponent, when you multiply them their arguments get added.

Aksakal
  • 55,939
  • 5
  • 90
  • 176