3

As I dig deeper than surface level in probability I'm starting to ask more questions I never thought about before.

There are a bunch of intertwined concepts that are quickly becoming confused in my head:

Probability Density Function/Probability Function: This tells us the probability of some value $x$ being equal to (in the case of a discrete distribution) or bounded by some $(a, b]$ in the event of a continuous distribution. Given that $P(a \le x \le a) = 0$ to use a continuous PDF we need some bounds for calculation.

Distribution Function defines $P(X \le x)$ which can also be defined as the integral from $-\inf$ to $x$ of the density function $f_x$: $\int \limits_{-\inf}^xf_x(x)dx$.

The density function's integral from negative infinity to infinity must be 1 - that is the area under the curve of the density function must be equal to 1.

So far so good (I think). Reading through the wikipedia entry for Probability Density Functions though it has some strange language that leads me to confusion.

The standard normal distribution has probability density $f_x(x) = \frac{1}{\sqrt{2\pi}}e^{-\frac{x^2}{2}}$

This density function is common knowledge and can be used to determine the probability of an event occurring.

The part where I am confused (and this leads to the title question) is the term

the standard normal distribution has probability density...

The implication here, of course, being the standard normal distribution being something unrelated to probability.

Suppose I draw a number from a standard normal distribution in python:

>>> import numpy as np
>>> np.random.normal(0, 1)
-0.8071008841914297

-0.8071008841914297 is clearly not a probability.

So when I draw a number from a normal distribution what is it actually? Taking a stab at it I would say:

"-0.8071008841914297" is the value (x-axis of the probability density function) of a normal distribution $N(0, 1)$. Plugging this value into the Distribution Function I would be using the integral above to find $P(X \le -0.8071008841914297)$.

But this still leaves me confused - what exactly then is a normal distribution? What does it look like? I can only find (and produce myself) images of the probability density function (this one stolen shamelessly from wikipedia):

enter image description here

Is this actually the normal distribution, and the probability density function is the function that defines the area under the curve?

EDIT:

Taking another stab at it:

A normal distribution is a way to describe the "look" of a continuous random variable. What I mean by "look" is how many of each possible value there are.

If we then assigned each $x$ in a continuous distribution a probability of occurring $p$, plotting these values would give us the graph above. The density function then tells us analytically the probability of any event we ask for occurring in this random variable.

CL40
  • 193
  • 1
  • 4
  • One of the best ways to resolve issues like these is to study the definitions. We have potentially useful threads on [random variables](https://stats.stackexchange.com/questions/50), [distribution functions](https://stats.stackexchange.com/questions/360798), [density functions](https://stats.stackexchange.com/questions/4220), and much more. – whuber Aug 09 '18 at 15:02

1 Answers1

1

You can imagine a process which produces numbers. This process is not deterministic, which means that when you run it it can produce different outputs for different executions. So there is uncertainty involved in each run. When you know how this uncertainty is distributed for all possible outcomes you say that you know the distribution of the process. The above is what we call a random variable.

The distribution of possible outcomes can be described in many ways. One is the cumulative distribution function or named usually distribution function. Another equivalent description is probability density function. They are equivalent because one can be obtained from the other. There are also other ways like characteristic function (under some conditions) and so on.

Drawing a sample from a distribution means observing a realization of a random variable which has assigned that distribution to possible outcomes. The outcome itself does not have probability assigned. The probability describes uncertainty. This uncertainty can be described by distribution using one form or the other but only before drawing the sample or before observing the outcome. Once you observe the the outcome the probability vanish. After observing you know the outcome and you are certain, no doubts on that. Of course you can always ask yourself which was the probability to observe some outcome, but that probability describes what was the situation before drawing.

rapaio
  • 6,394
  • 25
  • 45
  • So basically the _distribution_ is the underlying process that produces the numbers assigned to each _outcome_ in the outcome space described by the random variable $X$ and the various probability measures tell us how likely the numbers are. So in reality a random variable is a function that describes a _process_? A "normally distributed random variable" has values generated by a process described by $N(\mu, \sigma^2)$? – CL40 Aug 09 '18 at 06:40
  • The random variable is the process which has associated a distribution which describes the probability of each outcome produced by drawing from the random variable. The random variable is a process which when is executed / observed / run it produces outcomes according with the associated distribution. – rapaio Aug 09 '18 at 09:16