Questions tagged [dirichlet-distribution]

The Dirichlet distribution refers to a family of multivariate distributions, which are the generalization of the univariate beta distribution.

The Dirichlet distribution refers to a family of multivariate distributions, which are the generalization of the univariate beta distribution. The probability density function of an $n$ dimensional Dirichlet distribution is:

$$p(x_1,x_2,\ldots,x_n;\alpha_1,\alpha_2,\ldots,\alpha_n) = \frac{\Gamma(\sum_{i=1}^n \alpha_i)}{\prod_{i=1}^n \Gamma (\alpha_i)} \prod_{i=1}^n x_i^{\alpha_i-1}$$

The support of the function is $x_i \in (0,1)$ for all $i = 1, ..., n$, with the additional condition $\sum_{i=1}^n x_i = 1$. Note that $n \geq 2$, where $n$ is the number of categories.

The distribution is mostly used in Bayesian statistics as a prior for multinomial likelihood functions.

286 questions
41
votes
2 answers

What exactly is the alpha in the Dirichlet distribution?

I'm fairly new to Bayesian statistics and I came across a corrected correlation measure, SparCC, that uses the Dirichlet process in the backend of it's algorithm. I have been trying to go through the algorithm step-by-step to really understand what…
O.rka
  • 1,250
  • 4
  • 19
  • 30
38
votes
2 answers

Why is the Dirichlet distribution the prior for the multinomial distribution?

In LDA topic model algorithm, I saw this assumption. But I don't know why chose Dirichlet distribution? I don't know if we can use Uniform distribution over Multinomial as a pair?
ColinBinWang
  • 535
  • 1
  • 5
  • 5
33
votes
2 answers

Drawing from Dirichlet distribution

Let's say we have a Dirichlet distribution with $K$-dimensional vector parameter $\vec\alpha = [\alpha_1, \alpha_2,...,\alpha_K]$. How can I draw a sample (a $K$-dimensional vector) from this distribution? I need a (possibly) simple explanation.
user1315305
  • 1,199
  • 4
  • 14
  • 15
29
votes
1 answer

Can a Multinomial(1/n, ..., 1/n) be characterized as a discretized Dirichlet(1, .., 1)?

So this question is slightly messy, but I'll include colourful graphs to make up for that! First the Background then the Question(s). Background Say you have a $n$-dimensional multinomial distribution with equal probailites over the $n$ categories.…
23
votes
1 answer

Construction of Dirichlet distribution with Gamma distribution

Let $X_1,\dots,X_{k+1}$ be mutually independent random variables, each having a gamma distribution with parameters $\alpha_i,i=1,2,\dots,k+1$ show that $Y_i=\frac{X_i}{X_1+\cdots+X_{k+1}},i=1,\dots,k$, have a joint ditribution as…
23
votes
3 answers

Distribution of the largest fragment of a broken stick (spacings)

Let a stick of length 1 be broken in $k+1$ fragments uniformly at random. What is the distribution of the length of the longest fragment? More formally, let $(U_1, \ldots U_k)$ be IID $U(0,1)$, and let $(U_{(1)}, \ldots, U_{(k)})$ be the associated…
17
votes
1 answer

The input parameters for using latent Dirichlet allocation

When using topic modeling (Latent Dirichlet Allocation), the number of topics is an input parameter that the user need to specify. Looks to me that we should also provide a collection of candidate topic set that Dirichlet process has to sample…
15
votes
3 answers

Why does nobody use the Bayesian multinomial Naive Bayes classifier?

So in (unsupervised) text modeling, Latent Dirichlet Allocation (LDA) is a Bayesian version of Probabilistic Latent Semantic Analysis (PLSA). Essentially, LDA = PLSA + Dirichlet prior over its parameters. My understanding is that LDA is now the…
15
votes
1 answer

What is the expected value of modified Dirichlet distribution? (integration problem)

It is easy to produce a random variable with Dirichlet distribution using Gamma variables with the same scale parameter. If: $ X_i \sim \text{Gamma}(\alpha_i, \beta) $ Then: $ \left(\frac{X_1}{\sum_j X_j},\; \ldots\; , \frac{X_n}{\sum_j X_j}\right)…
15
votes
3 answers

Multinomial-Dirichlet model with hyperprior distribution on the concentration parameters

I will try to describe the problem at hand as general as possible. I am modeling observations as a categorical distribution with a parameter probability vector theta. Then, I assume the parameter vector theta follows a Dirichlet prior distribution…
14
votes
2 answers

Purpose of Dirichlet noise in the AlphaZero paper

In DeepMind's AlphaGo Zero and AlphaZero papers, they describe adding Dirichlet noise to the prior probabilities of actions from the root node (board state) in Monte Carlo Tree Search: Additional exploration is achieved by adding Dirichlet noise to…
monk
  • 395
  • 2
  • 10
12
votes
2 answers

Laplace smoothing and Dirichlet prior

On the wikipedia article of Laplace smoothing (or additive smoothing), it is said that from a Bayesian point of view, this corresponds to the expected value of the posterior distribution, using a symmetric Dirichlet distribution with parameter…
11
votes
1 answer

Dirichlet posterior

I have a question about the Dirichlet posterior distribution. Given a multinomial likelihood function it's known that the posterior is $Dir({\alpha_i + N_i})$, where $N_i$ is the number of times we've seen $i^{th}$ observation. What happens if we…
Max
  • 111
  • 2
11
votes
0 answers

Is sparsity of topics a necessary condition for latent Dirichlet allocation (LDA) to work

I have been playing with the hyper-parameters of the latent Dirichlet allocation (LDA) model and am wondering how sparsity of topic priors play a role in inference. I have not performed these experiments on real data, but on simulated data. I…
kedarps
  • 2,902
  • 2
  • 19
  • 30
10
votes
2 answers

The meaning of representing the simplex as a triangle surface in Dirichlet distribution?

I'm reading from a book that introduces the Dirchilet distribution and then presented figures about it. But I was not really able to understand those figures. I attached the figure here at the bottom. What I don't understand are the meanings of the…
Jack Twain
  • 7,781
  • 14
  • 48
  • 74
1
2 3
19 20