2

I encountered the maximum likelihood method. It says that the common probability density function of sample $X_1,\ldots,X_n$ having distribution $f(x;\theta )$ with observed values $x_1,\ldots,x_n$ is $f(x_1,\ldots,x_n;\theta )=\prod_i f(x_i;\theta)$.

Now what does the semicolon means in the definition of $f$? And is the mapping from $\mathbb{R}^n$ to $\mathbb R$ or from $\mathbb{R}^{n+1}$ to $\mathbb R$? Finally, how is it possible that the domain of $f$ changes from $\mathbb{R}^n$ or $\mathbb{R}^{n+1}$ to $\mathbb{R}$ or $\mathbb{R}^2$?

student
  • 33
  • 5
  • 1
    Closely related (and highly voted): http://stats.stackexchange.com/questions/29682/how-to-rigorously-define-the-likelihood, http://stats.stackexchange.com/questions/31238/what-is-the-reason-that-a-likelihood-function-is-not-a-pdf/31241#31241, http://stats.stackexchange.com/questions/2641/what-is-the-difference-between-likelihood-and-probability/2647#2647. – whuber Feb 01 '13 at 19:06

1 Answers1

2

The semicolon notation $f(x_1,...,x_n;\theta)$ means the conditional density of $x_1,...,x_n$ given $\theta$ with $\theta$ being the parameter that you want to estimate with the ML method. Instead of the semicolon you often find $f(x_1,...,x_n|\theta)$ which means the same thing.

The term $f(x_1,...,x_n;\theta)=\prod_i f(x_i;\theta)$ implies that the observations $x_1,...,x_n$ are mutually independent and identically distributed. This implies that the joint PDF required for the ML method can be written as the product of each individual PDF. This is obviously not generally true for every problem and also not a requirement for usage of the ML algorithm.

jan
  • 121
  • 2
  • 2
    The semi-colon ; is to be preferred to the vertical bar | as the later is usually reserved to conditioning in conditional probability. From a classical perspective, $\theta$ is an unknown parameter driving the distribution of the observations, not a random variable. If you instead follow a Bayesian approach, then the conditioning sign does make sense as $\theta$ becomes a random variable. – Xi'an Feb 02 '13 at 13:39