Estimation by maximizing the posterior density function
See https://en.wikipedia.org/wiki/Maximum_a_posteriori_estimation
Questions tagged [map-estimation]
48 questions
11
votes
3 answers
MAP is a solution to $L(\theta) = \mathcal{I}[\theta \ne \theta^{*}]$
I have come across these slides (slide # 16 & #17) in one of the online courses. The instructor was trying to explain how Maximum Posterior Estimate(MAP) is actually the solution $L(\theta) = \mathcal{I}[\theta \ne \theta^{*}]$, where $\theta^{*}$…
honeybadger
- 1,262
- 8
- 23
6
votes
1 answer
How do we derive the conditional mode as the solution to linear regression, for uniform cost function?
I know that if the cost functions are respectively the least squares ($L^2$) and the absolute deviation ($L^1$), the solution to linear regression is the conditional mean and the conditional median respectively. To see this, a simple method will be…
Bravo
- 515
- 5
- 9
4
votes
1 answer
How to prove that the posterior of the regression coefficients $\mathbf{w}$ is roughly gaussian in MAP regularized logistic regression?
The logistic regression model is
$$
p(y=\pm 1 \mid \mathbf{x}, \mathbf{w})=\sigma\left(y \mathbf{w}^{\mathrm{T}} \mathbf{x}\right)=\frac{1}{1+\exp \left(-y \mathbf{w}^{\mathrm{T}} \mathbf{x}\right)}
$$
It can be used for binary classification or…
Germania
- 224
- 10
4
votes
1 answer
Why should MAP be invariant under reparameterization?
I learned why MAP suffers from being reparametrization invariance while MLE not from this answer, but I don't know why reparametrization invariance even matters? What is the non-linear mapping concretely and why we do the non-linear mapping depicted…
Lerner Zhang
- 5,017
- 1
- 31
- 52
3
votes
1 answer
Bayesian MAP Estimates
Suppose you have a simple linear regression problem (y = bo + b1x) and you decide to use Bayesian Estimation to estimate the value pf bo and b1.
Using Bayesian Estimation, you obtain a list of different acceptable values for bo and b1. For instance,…
stats_noob
- 5,882
- 1
- 21
- 42
3
votes
1 answer
Why is MAP and ML widely accepted?
(ML as in Maximum Likelihood and MAP as in Maximum A-posteriori)
I'm going trough a course book on my own, and without really having peers to talk to I'm turning to stack exchange with these rather rudimentary question, I can't tell if I'm over…
Cookie
- 51
- 4
3
votes
3 answers
Differences between MLE and MAP estimators
Generally speaking, what are the differences between an MLE and a MAP estimator?
If I wanted to improve the performance of a model, how would these differences come into play? Are there specific assumptions about the model or the data that would…
MLNewbie
- 101
- 8
3
votes
1 answer
Does it make sense to condition on fixed values of some parameters before doing MCMC on the other parameters?
I have a Bayesian model with a large number of parameters (around 50), and as usual my goal is to infer the posterior distribution for the parameters, with MCMC.
However, I am only interested in the full posterior distribution for 5 of the…
segej733
- 33
- 2
3
votes
0 answers
Bias correcting penalized maximum likelihood / maximum a posteriori estimates
Suppose an estimator $\hat\theta_T$ is defined as the value of $\theta$ maximizing:
$$\sum_{t=1}^T{l(y_t|\theta)}+\mu_T g(\theta),$$
where $l(y_t|\theta)$ is the log-likelihood of observation $t$, $\mu_T$ determines the strength of penalization…
cfp
- 425
- 2
- 10
3
votes
3 answers
Bayesian parameter estimation with proportion data
I am trying to do a Bayesian analysis using a model that comes from the literature in non-Bayesian form: $y = \Phi\Bigg(\frac{1}{\alpha} * log(A/\beta)\Bigg)$. Because the model uses the function $\Phi$ its outcome is in the range $0-1$. My…
Jens de Bruijn
- 87
- 6
3
votes
2 answers
What is an example of a transformation on a posterior distribution such that the MAP estimate will be non-invariant?
Suppose that we have a posterior distribution $p(\theta\mid y)$ and we wish to define a transformation on $\theta$ such that $\phi = f(\theta)$. I know that generally such transformations will not affect the MLE as it is on the data space, but will…
user321627
- 2,511
- 3
- 13
- 49
3
votes
1 answer
The explanation for the need to compute (rather then optimize) the posterior of latent variables
The most common usage of the variational inference looks like to be in computing the marginal distribution $P(X)$ in the denominator of the Bayes formula when computing the posterior probability of the hidden variables, $P(Z|X)$. This is likely a…
user5054
- 1,259
- 3
- 13
- 31
2
votes
1 answer
Bayes prior in MAP estimation corresponding to $\ell^0$ penalization
I gather that in the context of penalized least squares, we can interpret a penalty term as corresponding to a prior $\pi(\beta)\propto \exp\{-\text{pen}\}.$
Is this also true for $\ell^0$ regularization,i.e. $\pi(\beta)\propto…
Golden_Ratio
- 343
- 7
2
votes
1 answer
Why maximum a posterior, not maximum posterior?
Is the additional "a" mean that different priors may lead to different posterior, MAP is a result of many possible results? And similar to MLE, why the abbreviation of maximum a posterior estimation is MAP, not MPE or MAPE? Thanks
Alex
- 31
- 4
2
votes
1 answer
Parameters in Naive Bayes
This is from https://scikit-learn.org/stable/modules/naive_bayes.html
In the last line it says
and we can use Maximum A Posteriori (MAP) estimation to estimate
$P(y)$ and $P(x_i|y)$; the former is then the relative frequency of
class $y$ in the…
Esha
- 21
- 2