Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [empirical-bayes]

Procedures for statistical inference in which the prior distribution is estimated from the data.

49 questions
27
votes
1 answer

How is empirical Bayes valid?

So I just finished reading a great book Introduction to Empirical Bayes. I thought that the book was great, but building priors from the data felt wrong. I was trained that you come up with an analysis plan then you collect data then you test the…
Alex
  • 888
  • 9
  • 24
24
votes
2 answers

Cross-validation vs empirical Bayes for estimating hyperparameters

Given a hierarchical model $p(x|\phi,\theta)$, I want a two stage process to fit the model. First, fix a handful of hyperparameters $\theta$, and then do Bayesian inference on the rest of the parameters $\phi$. For fixing the hyperparameters I am…
Memming
  • 1,570
  • 12
  • 23
18
votes
0 answers

Empirical Bayes (In)Admissibility

Most of the time, sticking to a pure Bayesian approach to statistics with proper priors, leads to admissible estimators. Nevertheless, there is a good reason to use Empirical Bayes in many cases, and the frequentists are enjoying better accuracy…
Cagdas Ozgenc
  • 3,716
  • 2
  • 29
  • 55
14
votes
1 answer

hierarchical Bayesian models vs. empirical Bayes

Would you consider the HBM vs EB to be two alternatives in which the hyperparameters are "in the game" of being sampled/estimated/etc.? There is clearly a connection between these two. Would you consider HBM more "fully Bayesian" than EB? Is there…
singelton
  • 652
  • 5
  • 13
14
votes
1 answer

Is there a connection between empirical Bayes and random effects?

I recently happened to read about empirical Bayes (Casella, 1985, An introduction to empirical Bayes data analysis) and it looked a lot like random effects model; in that both have estimates shrunken to global mean. But I have not read it…
anonymous
  • 141
  • 1
  • 3
6
votes
2 answers

Can "cross-validation" be used to choose a prior?

To be clear, I doubt I am using the term "cross-validation" correctly here; what I am suggesting also seems similar to "boot-strapping" and "hyperparameter tuning". Terminology is not my strength. Let's say we have a data set with $20$ observations,…
Chill2Macht
  • 5,639
  • 4
  • 25
  • 51
6
votes
1 answer

Using empirical Bayesian estimation (Gamma-Poisson) to analyze high arrival counts (n ~= 5000)

Here's a problem I'm currently working on, as well as the empirical Bayesian approach I'm using. I'd like to make sure my approach is grounded in solid statistical theory. I have a set of entities $e=e_1,e_2,...,e_N$, as well as arrival counts at…
achompas
  • 181
  • 5
6
votes
1 answer

Bootstrapping the data to set up a prior

I am using a Gaussian model with a conjugate Normal-Inverse-Wishart (NIW) prior, as described here. The advantage of this approach is that the marginal likelihood $p(y)$, which is what I am interested in, is available in closed form. My problem is…
Matteo Fasiolo
  • 3,134
  • 2
  • 20
  • 29
5
votes
1 answer

Admissible Empirical Bayes Examples

I would like to hear about a few simple empirical bayes estimators that are admissible for high (i.e. at least 3) dimensional parameter space. What are some textbook lollipop examples to study for beginners with easy derivations?
Cagdas Ozgenc
  • 3,716
  • 2
  • 29
  • 55
5
votes
2 answers

Is appropriate to use empirical Bayes (EB) in this way?

Background. I have data from a study where participants make a series of judgments (a series of decisions with a binomial outcome, either $y=1$ or $y=0$). I have a model of the underlying decision-making process, which has free parameters that can…
matteo
  • 2,631
  • 11
  • 19
5
votes
1 answer

Hyper-parameter estimation for Beta-Binomial Empirical Bayes

I am reading a paper Illustrating empirical Bayes methods and in the paper the author uses method of moments to derive the value of an estimate. In equation 17 the author gives the following marginal distribution $$m(y_i\vert \lambda) = {n\choose…
Alex
  • 888
  • 9
  • 24
4
votes
0 answers

Why Are Empirical Bayes Methods Not Considered "Controversial"?

I was reading about Empirical Bayesian Methods and came across the following: My Question: As this text explains, I have often heard that the priors used in Bayesian Methods should be decided prior to seeing any data - however, it seems that…
stats_noob
  • 5,882
  • 1
  • 21
  • 42
4
votes
1 answer

Using the MLE to select the prior distribution...empirical Bayes?

It was requested that I read the following article for work: https://support.sas.com/resources/papers/proceedings15/1400-2015.pdf In Case II, the author starts by doing two things: First, he computes the maximum likelihood estimator for the PD…
DavidSilverberg
  • 739
  • 6
  • 18
4
votes
2 answers

Accounting for uncertain information (few observations) in a prior (empirial Bayes)

I did not really know how to choose an adequate title for this question, so please feel free to change it. I have a weird case wherein frequentist and Bayesian philosophies come together. I am dealing with a data set of observations on persons who…
Damiaan Reijnaers
  • 63
  • 6
4
votes
1 answer

Likelihood of linear mixed effects model

Consider the following model $$\left \{ \begin{array}{l} y_i = x_i\beta + z_ib + \varepsilon_i,\\\\ b_i \sim \mathcal N(0, \Sigma), \quad \varepsilon_i \sim \mathcal N(0, \sigma^2), \end{array} \right.$$ where \begin{equation} \Sigma =…
JLee
  • 813
  • 3
  • 12
1
2 3 4