Is it possible that two Random Variables from the same distribution family have the same expectation and variance, but different higher moments?

Question

I was thinking about the meaning of location-scale family. My understanding is that for every $X$ member of a location scale family with parameters $a$ location and $b$ scale, then the distribution of $Z =(X-a)/b$ does not depend of any parameters and it's the same for every $X$ belonging to that family.

So my question is could you provide an example where two random from the same distribution family are standardized but that does not results in a Random Variable with the same distribution?

Say $X$ and $Y$ come from the same distribution family (where with family I mean for example both Normal or both Gamma and so on ..). Define:

$Z_1 = \dfrac{X-\mu}{\sigma}$

$Z_2 = \dfrac{Y-\mu}{\sigma}$

we know that both $Z_1$ and $Z_2$ have the same expectation and variance, $\mu_Z =0, \sigma^2_Z =1$.

But can they have different higher moments?

My attempt to answer this question is that if the distribution of $X$ and $Y$ depends on more than 2 parameters than it could be. And I am thinking about the generalized $t-student$ that has 3 parameters.

But if the number of parameters is $\le2$ and $X$ and $Y$ come from the same distribution family with the same expectation and variance, then does it mean that $Z_1$ and $Z_2$ has the same distribution (higher moments)?

Answer 1

If you want an example which is an "officially named parameterized distribution family, you can look into the generalized gamma distribution, https://en.wikipedia.org/wiki/Generalized_gamma_distribution. This distribution family has three parameters, so you can fix mean and variance and still have freedom to vary higher moments. From the wiki page, the algebra do not look inviting, I would rather to do it numerically. For statistical applications, search this site for gamlss, which is an extension of gam (generalized additive models, in itself a generalization of glm's) which have parameters for "location, scale and shape".

Another example is the $t$-distributions, extended to be a location-scale family. Then the third parameter will be the degrees of freedom, which will wary the shape for a fixed location and scale.

Answer 2

There is an infinite number of distributions with mean zero and variance one, hence take $\epsilon_1$ distributed from one of these distributions, say the $\mathcal{N}(0,1)$, and $\epsilon_2$ from another of these distributions, say the Student's $t$ with 54 degrees of freedom rescaled by $\sqrt\frac{1}{3}$ so that its variance is one, then $$X=\mu+\sigma\epsilon_1\qquad\text{and}\qquad Y=\mu+\sigma\epsilon_2$$ enjoy the properties you mention. The "number" of parameters is irrelevant to the property.

Obviously, if you set further rules to the definition of this family, like stating for instance that there exists a fixed density $f$ such that the density of $X$ is $$\frac{1}{\sigma^d} f(\{x-\mu\}/\sigma)$$ you may end up with a single possible distribution.

Answer 3

There is apparently some confusion as to what a family of distributions is and how to count free parameters versus free plus fixed (assigned) parameters. Those questions are an aside that is unrelated to the intent of the OP, and of this answer. I do not use the word family herein because it is confusing. For example, a family according to one source is the result of varying the shape parameter. @whuber states that A "parameterization" of a family is a continuous map from a subset of ℝ$^n$, with its usual topology, into the space of distributions, whose image is that family. I will use the word form which covers both the intended usage of the word family and parameter identification and counting. For example the formula $x^2-2x+4$ has the form of a quadratic formula, i.e., $a_2x^2+a_1x+a_0$ and if $a_1=0$ the formula is still of quadratic form. However, when $a_2=0$ the formula is linear and the form is no longer complete enough to contain a quadratic shape term. Those who wish to use the word family in a proper statistical context are encouraged to contribute to that separate question.

Let us answer the question "Can they have different higher moments?". There are many such examples. We note in passing that the question appears to be about symmetric PDFs, which are the ones that tend to have location and scale in the simple bi-parameter case. The logic: Suppose there are two density functions with different shapes having two identical (location, scale) parameters. Then there is either a shape parameter that adjusts shape, or, the density functions have no common shape parameter and are thus density functions of no common form.

Here, is an example of how the shape parameter figures into it. The generalized error density function and here, is an answer that appears to have a freely selectable kurtosis.

By Skbkekas - Own work, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=6057753

The PDF (A.K.A. "probability" density function, note that the word "probability" is superfluous) is $$\dfrac{\beta}{2\alpha\Gamma\Big(\dfrac{1}{\beta}\Big)} \; e^{-\Big(\dfrac{|x-\mu|}{\alpha}\Big)^\beta}$$

The mean and location is $\mu$, the scale is $\alpha$, and $\beta$ is the shape. Note that it is easier to present symmetric PDFs, because those PDFs often have location and scale as the simplest two parameter cases whereas asymmetric PDFs, like the gamma PDF, tend to have shape and scale as their simplest case parameters. Continuing with the error density function, the variance is $\dfrac{\alpha^2\Gamma\Big(\dfrac{3}{\beta}\Big)}{\Gamma\Big(\dfrac{1}{\beta}\Big)}$, the skewness is $0$, and the kurtosis is $\dfrac{\Gamma\Big(\dfrac{5}{\beta}\Big)\Gamma\Big(\dfrac{1}{\beta}\Big)}{\Gamma\Big(\dfrac{3}{\beta}\Big)^2}-3$. Thus, if we set the variance to be 1, then we assign the value of $\alpha$ from $\alpha ^2=\dfrac{\Gamma \left(\dfrac{1}{\beta }\right)}{\Gamma \left(\dfrac{3}{\beta }\right)}$ while varying $\beta>0$, so that the kurtosis is selectable in the range from $-0.601114$ to $\infty$.

That is, if we want to vary higher order moments, and if we want to maintain a mean of zero and a variance of 1, we need to vary the shape. This implies three parameters, which in general are 1) the mean or otherwise the appropriate measure of location, 2) the scale to adjust the variance or other measure of variability, and 3) the shape. IT TAKES at least THREE PARAMETERS TO DO IT.

Note that if we make the substitutions $\beta=2$, $\alpha=\sqrt{2}\sigma$ in the PDF above, we obtain $$\frac{e^{-\frac{(x-\mu )^2}{2 \sigma ^2}}}{\sqrt{2 \pi } \sigma }\;,$$

which is a normal distribution's density function. Thus, the generalized error density function is a generalization of the normal distribution's density function. There are many ways to generalize a normal distribution's density function. Another example, but with the normal distribution's density function only as a limiting value, and not with mid-range substitution values like the generalized error density function, is the Student's$-t$ 's density function. Using the Student's$-t$ density function, we would have a rather more restricted selection of kurtosis, and $\textit{df}\geq2$ is the shape parameter because the second moment does not exist for $\textit{df}<2$. Moreover, df is not actually limited to positive integer values, it is in general real $\geq1$. The Student's$-t$ only becomes normal in the limit as $\textit{df}\rightarrow\infty$, which is why I did not choose it as an example. It is neither a good example nor is it a counter example, and in this I disagree with @Xi'an and @whuber.

Let me explain this further. One can choose two of many arbitrary density functions of two parameters to have, as an example, a mean of zero and a variance of one. However, they will not all be of the same form. The question however, relates to density functions of the SAME form, not different forms. The claim has been made that which density functions have the same form is an arbitrary assignment as this is a matter of definition, and in that my opinion differs. I do not agree that this is arbitrary because one can either make a substitution to convert one density function to be another, or one cannot. In the first case, the density functions are similar, and if by substitution we can show that the density functions are not equivalent, then those density functions are of different form.

Thus, using the example of the Student's$-t$ PDF, the choices are to either consider it to be a generalization of a normal PDF, in which case a normal PDF has a permissible form for a Student's$-t$'s PDF, or not, in which case the Student's$-t$ 's PDF is of a different form from the normal PDF and thus is irrelevant to the question posed.

We can argue this many ways. My opinion is that a normal PDF is a sub-selected form of a Student's$-t$ 's PDF, but that a normal PDF is not a sub-selection of a gamma PDF even though a limiting value of a gamma PDF can be shown to be a normal PDF, and, my reason for this is that in the normal/Student'$-t$ case, the support is the same, but in the normal/gamma case the support is infinite versus semi-infinite, which is the required incompatibility.

Answer 4

I think you are asking whether two random variables coming from the same location-scale family can have the same mean and variance, but at least one different higher moment. The answer is no.

Proof: Let $X_1$ and $X_2$ be two such random variables. Since $X_1$ and $X_2$ are in the same location-scale family, there exist a random variable $X$ and real numbers $a_1>0, a_2>0, b_1, b_2$ such that $X_1 \stackrel{d}{=} a_1 X + b_1$ and $X_2 \stackrel{d}{=} a_2 X + b_2$. Since $X_1$ and $X_2$ have the same mean and variance, we have:

1. $E[X_1] = E[X_2] \implies a_1 E[X] + b_1 = a_2 E[X] + b_2$.
2. $\operatorname{Var}[X_1] = \operatorname{Var}[X_2] \implies a_1^2 \operatorname{Var}[X] = a_2^2 \operatorname{Var}[X]$.

If $\operatorname{Var}[X] = 0$, then $X_1=E[X_1]=X_2=E[X_2]$ with probability $1$, and hence the higher moments of $X_1$ and $X_2$ are all equal. So we may assume that $\operatorname{Var}[X] \neq 0$. Using this, (2) implies that $|a_1|=|a_2|$. Since $a_1>0$ and $a_2>0$, we have in fact that $a_1=a_2$. In turn, (1) above now implies that $b_1=b_2$. We therefore have that: $$ E[X_1^k] = E[(a_1X+b_1)^k] = E[(a_2X+b_2)^k] = E[X_2^k], $$ for any $k$, i.e., all moments of $X_1$ and $X_2$ are all equal.

Answer 5

Since the question can be interpreted in multipe ways I will split this answer into two parts.

• A: distribution families.
• B: location-scale distribution families.

The problem with case A can be easily answered/demonstrated by many families with a shape parameter.

The problem with case B is more difficult since one and a half parameters seem to be sufficient to specify location and scale (location in $\mathbb{R}$ and scale in $\mathbb{R_{>0}}$), and the problem becomes whether two parameters can be used to encode (multiple) shapes in addition as well. This is not so trivial. We can easily come up with specific two parameter location scale families and demonstrate that you do not have different shapes, but it does not proof that this is a fixed rule for any two parameter location scale family.

A: Can two different distributions from the same 2 parameter distribution family have the same mean and variance?

The answer is yes and it can already be shown using one of the explicitly mentioned examples: the normalized Gamma distribution

Family of normalized gamma distributions

Let $Z = \frac{X-\mu}{\sigma}$ with $X$ a Gamma distributed variable. The (cumulative) distribution of $Z$ is as below:

$$F_Z(z;k) = \begin{cases} 0 & \quad \text{if} & z < -\sqrt{k}\\ \frac{1}{\Gamma(k)} \gamma(k, {z\sqrt{k}+k}) & \quad \text{if} & z \geq -\sqrt{k} \end{cases} $$

where $\gamma$ is the incomplete gamma function.

So here it is clearly the case that different $Z_1$ and $Z_2$ (distributions from the family of normalized gamma distributions) can have same mean and variance (namely $\mu=0$ and $\sigma=1$) but differ based on the parameter $k$ (often denoted 'shape' parameter). This is closely linked to the fact that the family of gamma distributions is not a location-scale family.

B: Can two different distributions from the same 2 parameter location-scale distribution family have the same mean and variance?

I believe that the answer is no if we consider only smooth families (smooth: a small change in the parameters will result in a small change of the distribution/function/curve). But that answer is not so trivial and when we would use more general (non-smooth) families then we can say yes, although these families only exist in theory and have no practical relevance.

Generating a location-scale family from a single distribution by translation and scaling

From any particular single distribution we can generate a location-scale family by translation and scaling. If $f(x)$ is the probability density function of the single distribution, then the probability density function for a member of the family will be

$$f(x;\mu,\sigma) = \frac{1}{\sigma}f(\frac{x-\mu}{\sigma})$$

For a location-scale family that can be generated in such way we have:

• for any two members $f(x;\mu_1,\sigma_1)$ and $f(x;\mu_2,\sigma_2)$ if their means and variances are equal, then $f(x;\mu_1,\sigma_1) = f(x;\mu_2,\sigma_2)$

Can for all two parameter location-scale families their member distributions be generated from a single member distribution by translation and scaling?

So translation and scaling can convert a single distribution into a location-scale family. The question is whether the reverse is true and whether every two parameter location-scale family (where the parameters $\theta_1$ and $\theta_2$ do not necessarily need to coincide with the location $\mu$ and scale $\sigma$) can be described by a translation and scaling of a single member from that family.

For particular two parameter location-scale families like the family of normal distributions it is not too difficult to show that they can be generated according to the process above (scaling and translating of single example member).

One may wonder whether it is possible for every two parameter location-scale family to be generated out of a single member by translation and scaling. Or a conflicting statement: "Can a two parameter location-scale family contain two different member distributions with the same mean and variance?", for which it would be necessary that the family is a union of multiple subfamilies that are each generated by translation and scaling.

Case 1: Family of generalized Students' t-distributions, parameterized by two variables

A contrived example occurs when we make some mapping from $R^2$ into $R^3$ (cardinality-of-mathbbr-and-mathbbr2) which allows the freedom to use two parameters $\theta_1$ and $\theta_2$ to describe a union of multiple subfamilies that are generated by translation and scaling.

Let's use the (three parameter) generalized Student's t-distribution:

$f(x;\nu,\mu,\sigma) = \frac{\Gamma \left( \frac{\nu + 1}{2} \right) }{\Gamma \left( \frac{\nu}{2} \right) \sqrt{\pi\nu}\sigma} \left(1 + \frac{1}{\nu} \left( \frac{x-\mu}{\sigma} \right)^2 \right)^{-\frac{\nu+1}{2}}$

with the three parameters changed as following $$\begin{array}{rcl} \mu &=& \tan (\theta_1)\\ \sigma &=& \theta_2\\ \nu &=& \lfloor 0.5+\theta_1/\pi \rfloor \end{array}$$

then we have

$f(x;\theta_1,\theta_2) = \frac{\Gamma \left( \frac{\lfloor 0.5+\theta_1/\pi \rfloor + 1}{2} \right) }{\Gamma \left( \frac{\lfloor 0.5+\theta_1/\pi \rfloor}{2} \right) \sqrt{\pi\lfloor 0.5+\theta_1/\pi \rfloor}\theta_2} \left(1 + \frac{1}{\lfloor 0.5+\theta_1/\pi \rfloor} \left( \frac{x-\tan(\theta_1)}{\theta_2} \right)^2 \right)^{-\frac{\lfloor 0.5+\theta_1/\pi \rfloor+1}{2}}$

which may be considered a two parameter location-scale family (albeit not very useful) that can not be generated by translation and scaling of only a single member.

Case 2: Location-scale families generated by negative scaling of a single distribution with nonzero skew

A less contrived example, than using this tan-function, is given by Whuber under the comments of Carl's answer. We can have a family $x \mapsto f(x/b + a)$ where flipping the sign of $b$ keeps the mean and variance unchanged but possibly changing the uneven higher moments. So this gives a bit more easily a two parameter location-scale family where members with the same mean and variance can have different higher order moments. This example from Whuber can be split into two subfamilies each of which can be generated out of a single member by translation and scaling.

Smooth families

If we try to make a single smooth two parameter distribution family (smooth: a small change in the parameters will result in a small change of the distribution/function/curve) by somehow making a composition of two or more families that are generated by translation and scaling, then we get into problems to have the two parameters cover both the variation of 'mean' and 'variance', as well as the third parameter 'shape'. A formal proof will have to go along the same lines as the answer to the question: Is there a smooth surjective function $f:\mathbb{R}^2 \mapsto \mathbb{R}^3$? (where the answer is no in the case of smooth, ie. infinitely differentiable, functions although there are continuous functions that would do the job such as Peano curves).

Intuition: Imagine there would be some parameters $\theta_1$, $\theta_2$ that describe the distributions in some location-scale distribution family and by which we can change the mean and variance as well as some other moments, then we should be able to express $\theta_1$, $\theta_2$, in terms of the mean $\mu$ and variance $\sigma$

$$\begin{array}{rcl} \theta_1 &= &f_{\theta_1}(\mu,\sigma) \\ \theta_2 &=& f_{\theta_2}(\mu,\sigma)\end{array}$$

but these need to be multiple valued functions and these can not make continuous transitions, the different values from $f_{\theta_1}(\mu,\sigma)$ for a particular $\mu$ and $\sigma$ are not continuous, and will not be able to model a continuous shape parameter.

I am actually not so sure about this final part. We could possibly use a space-filling curve (such as the Peano curve, if only we knew how to express coordinates on the curve to coordinates of the hypercube) to have a single parameter $\theta_1$ completely model multiple features like mean and variance, without giving up the property that a small change of the parameter $\theta_1$ is equivalent to a small change of the function $f(x;\theta_1)$ at every $x$