Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [nonparametric-bayes]

Bayesian methods for infinite dimensional parameter spaces.

82 questions
25
votes
3 answers

Is there a Bayesian approach to density estimation

I am interested to estimate the density of a continuous random variable $X$. One way of doing this that I learnt is the use of Kernel Density Estimation. But now I am interested in a Bayesian approach that along the following lines. I initially…
renrenthehamster
  • 549
  • 5
  • 12
15
votes
4 answers

Gaussian Processes: How to use GPML for multi-dimensional output

Is there a way to perform Gaussian Process Regression on multidimensional output (possibly correlated) using GPML? In the demo script I could only find a 1D example. A similar question on CV that tackles case of multidimensional input. I went…
steadyfish
  • 1,772
  • 2
  • 15
  • 30
12
votes
0 answers

Help me understand the Bayesian kernel density estimation (Sibisi and Skilling, 1996)

Sibisi and Skilling (1996, also mentioned in the 1997 paper) define Bayesian kernel density as $$ f(x) = \int dx' \,\phi(x')\, K(x, x') \tag{2} $$ Here the kernel $K$ is an assigned smooth function, possibly having a few width and shape…
Tim
  • 108,699
  • 20
  • 212
  • 390
11
votes
1 answer

Covariance matrix for Gaussian Process and Wishart distribution

I'm reading through this paper on Generalised Wishart Processes (GWP). The paper calculates the covariances between different random variables (following Gaussian Process) using squared exponential covariance function, i.e., $K(x,x') =…
steadyfish
  • 1,772
  • 2
  • 15
  • 30
10
votes
2 answers

PyMC for nonparametric clustering: Dirichlet process to estimate Gaussian mixture's parameters fails to cluster

Problem setup One of the first toy problems I wanted to apply PyMC to is nonparametric clustering: given some data, model it as a Gaussian mixture, and learn the number of clusters and each cluster's mean and covariance. Most of what I know about…
Ahmed Fasih
  • 255
  • 2
  • 14
10
votes
1 answer

Putting a prior on the concentration parameter in a Dirichlet process

Most of this is background, skip to the end if you already know enough about Dirichlet process mixtures. Suppose I am modeling some data as coming from a mixture of Dirichlet processes, i.e. let $F \sim \mathcal D(\alpha H)$ and conditional on $F$…
guy
  • 7,737
  • 1
  • 26
  • 50
10
votes
2 answers

Introductory textbook on nonparametric Bayesian models?

I'd like to wrap my head around this topic but learning from white-papers and tutorials is hard because there are many gaps which are usually filled in textbooks. If it is important I have relatively strong mathematical background as I did my Ph.D.…
Moonwalker
  • 389
  • 1
  • 13
10
votes
1 answer

Do Stochastic Processes such as the Gaussian Process/Dirichlet Process have densities? If not, how can Bayes rule be applied to them?

The Dirichlet Pocess and Gaussian Process are often referred to as "distributions over functions" or "distributions over distributions". In that case, can I meaningfully talk about the density of a function under a GP? That is, do the Gaussian…
snickerdoodles777
  • 655
  • 3
  • 13
9
votes
1 answer

What does it mean to integrate over a random measure?

I'm currently looking at a paper of Dirichlet process random effects model and the model specification is as follows: $$ \begin{align*}y_{i} &= X_{i}\beta + \psi_{i} + \epsilon_{i}\\ \psi_{i} &\sim G \\ G &\sim \mathcal{DP}\left(\alpha, G_{0}\right)…
Daeyoung Lim
  • 642
  • 3
  • 12
7
votes
2 answers

How adding covariance noise in Gaussian processes to prevent overfitting?

I am told, in Gaussian Processes, adding covariance function noise to others, say SEiso or Materns, cause a better result, since it prevents from over fitting. I appreciate if someone could put more light on it !
Areza
  • 1,058
  • 2
  • 11
  • 30
7
votes
2 answers

Chinese Restaurant process (CRP)

I am trying to understand the Chinese Restaurant process (CRP) and Weighted Chinese Restaurant process (WCRP) described in a research paper "Automatic Discovery of Cognitive Skills"- Robert V. Lindsey, Mohammad Khajah, Michael C. Mozer to Improve…
Nakshu
  • 79
  • 1
7
votes
3 answers

Books for learning non parametric Bayesian model

Having studied parametric Bayesian statistics during the two last years, I plan to begin to self-study non parametric Bayesian model during this summer and look for recommendations. I would like the book(s) to cover both the theoritical aspects and…
peuhp
  • 4,622
  • 20
  • 38
6
votes
2 answers

Understanding the construction of Dirichlet process

I'm trying to understand the construction process of DP, however, with little background in measure theory, the original papers are hard to read, but I believe the ideas behind these papers can be followed. Let $y_1, y_2, \ldots, y_n$ be i.i.d.…
fishiwhj
  • 422
  • 4
  • 11
6
votes
2 answers

What are some applications of Chinese restaurant processes?

What are some applications of Chinese restaurant processes? I'm trying to learn a bit about non-parametric Bayesian methods, starting with Dirichlet processes and CRPs, but all the tutorials I've found are about theory, without describing any…
raegtin
  • 9,090
  • 12
  • 48
  • 53
5
votes
1 answer

understanding of effect of $\alpha$ in Dirichlet distribution

When reading the topic modeling tutorial written by Blei, KDD 2011 tutorial I was confused about a set of diagrams which aim to show the effect of $\alpha$ in Dirichlet distribution. For example, for the plot with $\alpha=1$, what am I suppose to…
user3269
  • 4,622
  • 8
  • 43
  • 53
1
2 3 4 5 6