13

My stats course just taught me that a discrete random variable has a finite number of options ... I hadn't realized that. I would have thought, like a set of integers, it could be infinite. Googling and checking a several web pages, including a few from university courses, has failed to specifically confirm this; most sites do however say discrete random variables are countable - I suppose that means finitely numbered?

It is clear that continuous random variables are infinite even if (most?) often bounded.

But if discrete random variables have finite possibilities, what then is an infinite distribution of integers? It's neither discrete nor continuous? Is the question moot because variables either tend to be continuous & (by definition) infinite or discontinuous & finite?

James
  • 392
  • 3
  • 11
  • 10
    You should ask your stats course about geometric and poisson random variables – probabilityislogic Jun 13 '18 at 12:26
  • It's online, so limited feedback. You're suggesting they're a third (and fourth?) type of variable, rather than just(!) distributions? – James Jun 13 '18 at 12:40
  • 4
    A distribution is not a random variable--and ignoring that distinction has [confused many.](https://stats.stackexchange.com/questions/332227) A beautiful theorem of early 20th century mathematics, the [Lebesgue decomposition theorem](https://en.wikipedia.org/wiki/Lebesgue%27s_decomposition_theorem#Refinement), shows how to conceive of all distribution functions as comprised of three distinct kinds: "continuous" (which are further subdivided into absolutely continuous and continuous but not a.c.) and "discrete." – whuber Jun 13 '18 at 13:09
  • 2
    not a good course you're taking i'm afraid – Aksakal Jun 13 '18 at 14:04
  • To all the responses here, thank you (though some are over my head I will confess). I should probably refer to what triggered this question as on reviewing it I may have interpreted it incorrectly: a true/false question stating "A discrete random variable can take a finite number of distinct values" is considered true; with the explanation that the statement "is one of the key properties of a discrete random variable". If we surveyed farmers asking how many cattle they owned, it would be impossible to bound the number beforehand, it is theoretically infinite but discrete ... ? – James Jun 14 '18 at 10:46

3 Answers3

15

If that's what your course said, it's wrong.

While discrete distributions can have a finite number of possible outcomes, they are not required to; you can have a discrete distribution that has an infinite number of possible outcomes - the number of elements should be no more than countable.

A common example would be a geometric distribution; consider the number of tosses of a fair coin until you get a head. There's no finite upper bound on the number of tosses that may be needed. It may take 1 toss, or 2, or 3, or 100, or any other number.

A discrete distribution could be negative (consider the difference between two such geometrically-distributed random variables; it can be any positive or negative integer).

A discrete distribution doesn't have to be over the integers, though, like in my example. That's just a common situation, not a requirement.

Glen_b
  • 257,508
  • 32
  • 553
  • 939
  • So what's the actual condition that makes a distribution "discrete"? : ) – Matthew Drury Jun 13 '18 at 18:42
  • The condition is that it has Lebesgue measure zero, is it not, @matthewDrury?. Which in turn is equivalent to the distribution summing to one on at most a countable set. – Therkel Jun 13 '18 at 20:39
  • I must admit I don't know the canonical definitions. I'm curious as to the role of accumulation points in all this. – Matthew Drury Jun 13 '18 at 20:45
  • 1
    @Therkel I would think that a distribution over the Cantor Set would not be considered "discrete". – Acccumulation Jun 13 '18 at 20:59
  • After checking https://en.wikipedia.org/wiki/Countable_set I'm happy to accept this as the answer; the geometric distribution example is clear, and it seems to represent the consensus of responses contributed so far. – James Jun 14 '18 at 22:40
5

I'm writing an answer, with the perspective that I have only a very naive understanding of measure-theoretic probability (so, experts, please correct me!).

A (real-valued) random variable is a function $X: S \to \mathbb{R}$, where $S$ is a sample space.

$X$ is discrete if $X(S)$, the image of $S$ induced by $X$, is countable. $X$ is continuous if $X$ has an absolutely continuous CDF. (I don't know that much about absolutely continuous functions, so I can't elaborate on this point.)

However, not all random variables are only discrete or continuous. There are "mixed" random variables, where $X(s)$ has a CDF which is the sum of a step function and a continuous function with indicators.

You can also have random variables which are neither discrete nor continuous, such as the Cantor distribution.

Clarinetist
  • 3,761
  • 3
  • 25
  • 70
  • 1
    You actually know quite a lot about absolutely continuous distributions, because (almost by definition) an absolutely continuous distribution is one that has a density. There are continuous distributions that do not have densities: the archetypal example is the distribution induced by the [Cantor function](https://en.wikipedia.org./wiki/Cantor_function). – whuber Jun 13 '18 at 14:30
  • If the countable image has an accumulation point, would we still say its discrete? – Matthew Drury Jun 13 '18 at 18:43
  • 1
    @Matthew Yes. The example I mentioned in another comment (https://stats.stackexchange.com/a/104018/919), which clearly is discrete (every one of a countable number of values has a nonzero probability, so its distribution function consists of nothing but jumps) has the entire interval $[0,1]$ for its set of accumulation points. – whuber Jun 14 '18 at 01:52
3

To quote the wikipedia page on continuous and discrete variables:

If it [the variable] can take on two particular real values such that it can also take on all real values between them (even values that are arbitrarily close together), the variable is continuous in that interval

Therefore, a discrete random variable does not have to have a 'finite number of options', but there needs to be a non-infinitesimal gap between the possible values. This is the case with a distribution of integers, since the 'distance' between two neighboring integers is 1 and cannot be less than this. Therefore the variable is not continuous as it does not 'continue' within these gaps.

Edit: I know that there's probably better and/or more precise ways of answering this, but this is what helped me personally understand the difference.

deemel
  • 2,402
  • 4
  • 20
  • 37
  • 4
    Just fyi, for intuition the Wikipedia characterization might be OK, but for most other purposes it's not correct. One important aspect of "continuous random variable" that it omits (out of several) is that it depends on *the probabilities* of its values, not just on the set of values it can attain. Your characterization of "non-infinitesimal" gap unfortunately is incorrect. I give a counterexample at https://stats.stackexchange.com/a/104018/919 showing a discrete variable that assigns positive probabilities to *all* rational numbers between $0$ and $1.$ – whuber Jun 13 '18 at 13:05
  • 1
    Some authors have said that values that get arbitrarily close together are not discrete, but I must admit I find it odd (though perhaps I am missing something). An example is the distribution of the difference of square roots of two Poisson random variates (w.real applications: people do sometimes take square roots with variables thought to be Poisson to stabilize the variance and may be interested in whether pair-differences are centered at zero). Values may be arbitrarily close together with such a variate but they are always distinct (you can enumerate each one), ...ctd – Glen_b Jun 13 '18 at 23:04
  • ctd... and all the values it does take have positive probability. A simpler example would be the reciprocal $Y=1/X$ of a geometric random variate $X$ (number of trials form): for any $ε>0$ there are values closer together than that - so $X$ is discrete but $Y$ is not? Does anyone know of a good reason why this particular distinction (that values cannot be arbitrarily close together for it to be discrete) is drawn by some authors? Compare with the example of a discrete set ...ctd – Glen_b Jun 13 '18 at 23:04
  • ctd... given at Mathworld ([Discrete Set](http://mathworld.wolfram.com/DiscreteSet.html)): "Another example of an infinite discrete set is the set $\{1/n\,\text{ for all integers }n>1\}$" -- I'd have expected the definition of discrete random variable to not be inconsistent with that (aside some very compelling reason to make it inconsistent). – Glen_b Jun 13 '18 at 23:43
  • 3
    @Glen Those authors appear to confuse two different concepts of "discrete": one is the measure-theoretic idea discussed here and the other is the [topological concept](https://en.wikipedia.org/wiki/Discrete_space) in which each element of a discrete set $A$ in a topological space is contained within an open set having no other elements of $A$ in it. Although it's nice that a probability measure supported on any discrete subset of the real line will be discrete, the converse is not true: discrete measures need not be supported on discrete subspaces. – whuber Jun 14 '18 at 01:58
  • 1
    I suppose that's a mix up I had in my head. I'm a trained topologist, so discrete definitely rings in the topological context when I hear it. Thanks for clarifying @whuber. – Matthew Drury Jun 14 '18 at 02:31