12

Consider a random vector $X\equiv (X_1,...,X_L)$. Assume that each $X_l$ is continuously distributed with support $\mathbb{R}$, for $l=1,...,L$. Does this imply that also $X$ should be continuously distributed (although, not necessarily with support $\mathbb{R}^L$)?

I clarify that, by support of $X_l$, I intend the smallest closed set $\mathcal{X}$ such that $Pr(X_l\in \mathcal{X})=1$.

TEX
  • 266
  • 3
  • 19
  • 1
    [Wikipedia says that](https://en.wikipedia.org/wiki/Probability_distribution#Continuous_probability_distribution) "A continuous probability distribution is a probability distribution whose support is an uncountable set". Under this definition, the answer is trivially *yes*. Are you working with some other definition of a continuous distribution? – Stephan Kolassa Oct 05 '20 at 20:06
  • @StephanKolassa What about $(X,Y)$ for normal $X$ and binary $Y?$ – Dave Oct 05 '20 at 20:14
  • 3
    @Stephan Although the Wikipedia article is vague, it is evident the context of the quotation is a *univariate* probability distribution. The quotation is obviously incorrect in more than one dimension; take *e.g.* the random variable $(Z,Z)$ where $Z$ has a standard Normal distribution. Incidentally, the term "support" in the quotation is misleading. That is [usually taken to be](https://en.wikipedia.org/wiki/Support_(mathematics)#In_probability_and_measure_theory) the smallest closed set having unit probability; but there exist discrete distributions whose support is the entire real line. – whuber Oct 05 '20 at 20:20
  • 1
    @Dave A binary $Y$ can hardly be said to be _continuously distributed with support $\mathbb R$_ as the OP's hypotheses aver. – Dilip Sarwate Oct 05 '20 at 20:38
  • @DilipSarwate Certainly the binary variable isn’t continuous, but the bivariate distribution has support on an uncountable set, as the Wikipedia definition discusses. – Dave Oct 05 '20 at 20:54
  • @whuber - I'll bite. What is a simple example of a discrete distribution whose support is the entire real line? – Henry Oct 05 '20 at 21:23
  • @Henry For intuition, see https://stats.stackexchange.com/a/104018/919 which describes a discrete distribution whose support is the unit interval $(0,1].$ From it you can readily construct discrete distributions supported on the entire real line; e.g., by truncating the distribution to $(0,1)$ and continuously transforming that to the line; or -- for a totally different approach -- add any distribution supported on the integers to that one. – whuber Oct 05 '20 at 21:27
  • What is truly interesting in the current setting is that among the discrete bivariate distributions whose marginals are supported on the entire line, there are some supported on the entire plane and others whose support is *zero* dimensional: that is, a collection of isolated points! (Constructing the latter is an amusing exercise.) – whuber Oct 05 '20 at 21:28
  • 1
    @whuber: Thank you. I think what you are saying is that the closure of the rational numbers is the real numbers (not something I had thought about) and that it is possible to have a discrete distribution on the rational numbers (something I had previously thought about) – Henry Oct 05 '20 at 21:32
  • @Henry That's correct. Indeed, the real numbers are usually constructed from the rational numbers as a closure process, so by definition the rationals are dense in the reals. These fundamental topological distinctions explain why characterizations of discrete random variables in terms of the cardinalities of their support are doomed to fail: the distinction between countable and uncountable supports just doesn't get to the heart of the matter. – whuber Oct 05 '20 at 21:34
  • 2
    The Wikipedia definition seems incorrect. A mixed distribution will also have uncountable support. – John Coleman Oct 06 '20 at 09:38

2 Answers2

15

No, if the individual random variables are continuous and thus their marginal distributions can be described using pdfs, it is not necessarily the case that they enjoy a joint pdf. A standard counterexample to the OP's "claim" is when $X \sim N(0,1)$ and $Z$ is an independent discrete random variable taking on values $\pm 1$ with equal probability. Then, $Y = ZX \sim N(0,1)$ also, but $(X,Y)$ does not have a joint pdf (measured in units of probability mass per unit area). All the probability mass lies on the lines $y=x$ and $y=-x$ and since lines have zero area, $(X,Y)$ does not enjoy a joint pdf (measured in units of probability mass per unit area). See, for example, this answer by Macro.

Dilip Sarwate
  • 41,202
  • 4
  • 94
  • 200
  • Thanks. Your example seems close to the first example of the other answer. However, the other answers seems to say that there can be a proper pdf on the line $y=x$. Which one is correct? – TEX Oct 05 '20 at 23:59
  • 3
    There _is_ a pdf on the straight line but it is **not a joint pdf** which has units probability mass _per unit area_ but rather a _univariate_ pdf measured in units of probability mass _per unit **length** as measured along the straight line. Looked at another way, the joint CDF $F_{X,Y}(x,y)=P(X\leq x,Y\leq y)$ is not continuous everywhere in the plane, and so is not differentiable everywhere in the plane either, and so we cannot write $f_{X,Y}(x,y)=\frac{\partial^2}{\partial x \partial y}F_{X,Y}(x,y)$ as we can for jointly continuous random variables. – Dilip Sarwate Oct 06 '20 at 16:56
12

No, a very simple counterexample is $(Z, Z)$ where $Z \sim \mathcal{Norm}(0,1)$ where the marginals are standard normal but the joint distribution is concentrated on the diagonal line $y=x$. So the joint distribution do not have a density with respect to the Lebesgue measure on the plane, but it does indeed have a density with respect to the Lebesgue density on that line $y=x$.

Another simple example, but here the marginals have a density which is positive on the segment $[-1, 1]$. Let $(X,Y)$ have the uniform distribution on the unit circle, that is, we can represent it as $X=\cos(\theta), Y=\sin(\theta)$ where $\theta \sim \mathcal{Uniform}(0, 2\pi)$. I did a simulation in R:

enter image description here

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
  • Thanks, but in both examples the vector $X$ is continuosly distributed (although not with support equal to the plane). Hence it does not contradict my statement. – TEX Oct 05 '20 at 23:54
  • Also the first example seems to contradict with the other answer which says that there can't be a proper pdf on the line $y=x$ – TEX Oct 05 '20 at 23:55
  • 2
    A density is always *with respect to a base measure* There can be no density on $y=x$ *with respect to Leb Measure on the plane (as then the line has measure 0), but it can be *with respect to Leb measure on the line itself*. – kjetil b halvorsen Oct 05 '20 at 23:58
  • 3
    In both examples the vector $X$ is concentrated on a set with measure 0 with respect to Leb measure *on the plane*. So it is not absolutely continuous *with respect to that measure*. This makes clear that in these examples we need to be more precise with the terminology! – kjetil b halvorsen Oct 06 '20 at 00:01
  • See https://stats.stackexchange.com/questions/298293/absolutely-continuous-random-variable-vs-continuous-random-variable for details about absolute continuity! – kjetil b halvorsen Oct 06 '20 at 17:23