Gaussian-to-gaussian transformations

Question

It is well-known that when a linear transformation is applied to a normally-distributed random variable, the result is itself a normally-distributed random variable.

I am interested in the converse of this. If I apply an unknown transformation to a normally-distributed random variable, and find that the result is normally-distributed, does this imply that the transformation must be linear?

It is apparent that this cannot be true in the most general case, as one can invent pathological non-linear transformations that preserve the distribution while being non-linear, e.g. $$ x \to\begin{cases}-x & |x|<1\\x&\mathrm{otherwise}\end{cases} $$ will not affect a normal distribution centred on $x=0$. However, can the statement be rendered true by placing some (relatively weak) restrictions on the form of the transformation, e.g. that it is in some sense smooth and continuous?

Does https://stats.stackexchange.com/questions/200380/normal-distribution-existence-non-affine-invariant-transformation/200407#200407 answer your question? Or are you asking something different--and if so, how? — whuber, Feb 15 '19 at 15:38
@whuber That question proves that there are non-affine transformations that preserve normality. My interest is in whether it is possible to use preservation of normality to infer that a transformation must be affine, if we place some weak restrictions on the character of that transformation (e.g. smoothness, somehow defined). I think your answer hits the mark. — avid, Feb 19 '19 at 05:16

score 5 · Accepted Answer · edited Jun 11 '20 at 14:32

The thread at Normal Distribution Existence Non-affine Invariant Transformation? exhibits many non-affine transformations that map normally distributed variables into normally distributed variables, so the answer is in the negative.

(For the record, an affine transformation $f:\mathbb R\to \mathbb R$ is one of the form $f(x) = \mu + \sigma x$ for numbers $\mu$ and $\sigma.$ That is, its graph is a line. I will restrict this definition to positive $\sigma$ (positively sloping graphs) in order to obtain an easily stated result.)

We can, however, point to a simple condition that assures a positive answer, as suggested in the question. I will state it rather generally because that reveals its nature and it is no harder to prove the generalization.

Let $X$ be a continuous random variable supported on all the real numbers. (This implies its distribution function $F_X$ given by $F_X(x)=\Pr(X\le x)$ is a strictly increasing continuous function from $\mathbb R$ onto the interval $(0,1).$)

Denote by $\mathcal{F}[F_X]$ the set of distributions of all invertible affine transformations of $X$ (the "location-scale family" of $X$). Thus

$$\mathcal{F}[F_X] = \{F_{\mu+\sigma X}\mid \mu\in\mathbb{R},\,\sigma\gt 0\}.$$

Proposition

Let $f:\mathbb R\to\mathbb R$ be a continuous, increasing, one-to-one, measurable function for which $F_{f(X)}\in\mathcal{F}[F_X].$ Then $f$ is affine.

Proof

The measurability of $f$ is required to assure $Y=f(X)$ is a random variable.

By applying a preliminary affine transformation to $f(X)$ we may, without any loss of generality, assume $F_X=F_Y=F_{f(X)}.$

Let $y$ be any real number and compute

$$F_X(y) = F_Y(y) = F_{f(X)}(y) = \color{red}{\Pr(f(X) \le y) = \Pr\left(X\le f^{-1}(y)\right)} = F_X\left(f^{-1}(y)\right).$$

The first three and last equalities are all definitions. The remaining (penultimate) equality, shown in red, is justified because $f$ is one-to-one and increasing, whence its inverse $f^{-1}$ exists, is unique, and preserves inequalities.

Finally, comparing the left and right sides of the foregoing, observe that because $F_X$ is strictly increasing and continuous, equality can hold only when the arguments are equal: that is,

$$F_X(y) = F_X\left(f^{-1}(y)\right)\text{ implies } y = f^{-1}(y)$$

for all real $y.$ That means $f^{-1}$ (and therefore $f$) is the identity function. Accounting for the preliminary affine transformation that was applied, we conclude that the original transformation was affine, QED.

Readers might enjoy constructing counterexamples to the proposition when any of the conditions on $f$ are relaxed.

To answer the question: This applies to Normally distributed variables because they are a location-scale family of absolutely continuous variables with distributions supported on the entire real line.

Great answer! Can you explain where the assumption that $f$ is continuous is used in your Proposition? — Artem Mavrin, Feb 18 '19 at 01:50
Also, I think your conditions may be relaxed to just “$f$ is strictly increasing” since strictly increasing functions on $\mathbb{R}$ are automatically Borel-measurable (as are continuous functions, if that assumption is needed) — Artem Mavrin, Feb 18 '19 at 01:52
@Artem Thank you for pointing out those implications. The continuity is used to conclude $f$ is a surjection. Although it's not a necessary assumption here, I first introduced it in my considerations when pondering extensions to non-continuous random variables, and left it in. Obviously, though, if $f$ is not continuous, then $f(X)$ cannot be a continuous variable. — whuber, Feb 18 '19 at 12:54
@Whuber: Is it the case that affine transformations are the only ones that preserve normality? — MSIS, Feb 26 '20 at 01:40

Peter Leopold · Answer 2 · 2019-02-16T00:01:41.297

0

If $y=f(x), x,y\in\mathbb{R}^m$, then the condition that $x$ and $y$ are normally distributed is that the Jacobian of the transformation $f$ exists and is constant everywhere, with thanks to @henry for improvements.

Please see the comment by @whuber in the OP’s question, viz., Normal Distribution Existence Non-affine Invariant Transformation?”, for his detailed discussion of non-affine transformations that preserve normality.

edited Feb 16 '19 at 00:01

answered Feb 15 '19 at 05:44

Peter Leopold

1,653
7
21

1

You may need differentiability of $f$ for that statement (and something like continuity for the original statement) – Henry Feb 15 '19 at 08:40
This isn't true unless you are making a very narrow interpretation of the question. Please, then, explain what you understand the question to ask and be particularly careful of your quantifiers! – whuber Feb 15 '19 at 15:37
1. @whuber’s solution goes much deeper than mine & seems to be in the spirit of the OP’s question. Bravo. 2. as @Henry points out, a notion of continuity cannot be ignored, to wit: 3. CasellaBerger tell us the right continuity is all we need for a CDF. 4. Which is implicit in @whuber’s discussion. 5, after the pyrotechnics of the non-affine transformations, Y=f(X) will still satisfy the requirement that a differential unit of x will be homogeneously distorted in creating a differential unit of y. So when all is said and done, the overall Jacobian remains constant everywhere. – Peter Leopold Feb 15 '19 at 23:54
It is not true the Jacobian must be everywhere constant. I think you misundertand my post, Peter: it shows *only a few* examples. There are many transformations $f:\mathbb{R}\to\mathbb{R}$ for which $f(X)$ is standard Normal when $X$ is, are almost everywhere infinitely differentiable, *but have non-constant Jacobians.* For instance, writing $\Phi$ for the standard Normal CDF and $F_{\chi^2(1)}$ for the chi-squared CDF (with one degree of freedom), $$f(x)=\Phi^{-1}\left(F_{\chi^2(1)}(x^2)\right)$$ is differentiable at all nonzero $x,$ has a varying derivative, and preserves standard Normality. – whuber Feb 16 '19 at 19:47
In pondering this some more, I suspect what you might be trying to claim is that when $f:\mathbb{R}\to\mathbb{R}$ is *continuously differentiable* and $f(X)$ has a Normal distribution for some particular random variable $X$ that is normally distributed, then $f$ must be affine. I believe that *is* true. The crux of the demonstration is to show $f$ cannot have any critical points. – whuber Feb 16 '19 at 22:30
@whuber How I understand your post: take any normally-distributed $X \sim N(\mu,\sigma^2)$ and map it to the unit interval using $u \sim \Phi(X).$ Then perform any slice-n-dice operation on $u$. Then transform $u$ to $y \sim \Phi^{-1}(u)$. Since the unit interval is the great meeting place of CDFs, there is no reason why we can't pass $X$ through $Z=x^2 \sim \chi^2$and finally to Y. I hadn't thought of any of that, and I find it pretty interesting and enlightening in its own right. That the overall $f$ is affine is comforting. Right continuity of CDF should satisfy crit. pt. pblm. – Peter Leopold Feb 17 '19 at 17:01
The critical points apply to $f,$ not to the distribution. My "slice-and-dice" operation is intended only as an *illustration* of the many possibilities, not as an exhaustive account of all of them. The logic here is important: for instance, the example I gave above is not a slice-and-dice operation. I think you have grasped the fundamental idea, though: using a CDF and inverse CDF enables us to reduce the problem to finding probability-preserving transformations of the unit interval (at least for continuous variables). – whuber Feb 17 '19 at 17:06
@Whuber: If X is normal, what transformations other than affine preserve its normality? – MSIS Feb 26 '20 at 02:57
1

@MSIS See https://stats.stackexchange.com/a/200407/919. – whuber Feb 26 '20 at 13:39

Gaussian-to-gaussian transformations

2 Answers2

Proposition

Proof