Monte Carlo Integration and pdfs?

Question

Let's say I have an un-normalized probability density function $f(x)$, which is related to $\xi$ via $\xi = \frac{f}{c}$

I also have a sample set $S = \{x_i\}_{i=1}^n \sim \xi$ which is sampled from the normalized pdf $\xi$

Can $S$ then be used to determine the normalizing constant $c$?

That is, in a simple case taken from Wikipedia

$$p(x) = e^{-x^2/2}$$ so, $$\int_{-\infty}^\infty p(x)dx = \int_{-\infty}^{\infty}e^{-x^2/2}dx = \sqrt{2\pi} = c$$ and if the function $\phi(x)$ is defined as: $$\phi(x) = \frac{1}{\sqrt{2\pi}}p(x) = \frac{1}{\sqrt{2\pi}}e^{-x^2/2}$$ so that $$\int_{-\infty}^{\infty}\phi (x)dx = \int_{-\infty}^{\infty}\frac{1}{\sqrt{2\pi}}e^{-x^2/2}dx = 1$$

then $\frac{1}{\sqrt{2\pi}}$ is the normalizing constant of $p(x)$.

So in the case of the sampling set $S$, would I determine the approximate normalized pdf $\xi$ via a histogram (and potentially curve fitting) and compare it to $f(x)$?

Is that the same as $c = \int_{-\infty}^{\infty} p(x)dx$ from Wikipedia?

Goal: Find $c$ by using $S$, is this possible?

Answer 1

This is a fairly common problem for Bayesian statistics where the posterior distribution$$p(\theta|x)=\dfrac{f(x|\theta)\pi(\theta)}{\int_\Theta > f(x|\theta)\pi(\theta)\,\text{d}\theta}=\dfrac{f(x|\theta)\pi(\theta)}{m(x)}$$most often involves an intractable normalising constant $m(x)$.

One of my answers to earlier questions on that topic lists a range of solutions, based on simulation. A book-length entry is found in Chen, Shao and Ibrahim (2001).

Reverting to the question and its notations, if the only available material is provided by the sample$$\mathfrak S = \{x_i\}_{i=1}^n \sim \xi$$ with no further access to simulation, contrary to the previous answer, a range of solutions can be found by a reverse version of the importance sampling method, namely that, for any density function $\alpha(\cdot)$ [with the same support as $f(\cdot)$, at most], the following general identity holds:$$\mathbb{E}_\xi\left[\frac{\alpha(X)}{f(X)}\right]=\int_{\mathfrak X} \dfrac{\alpha(x)}{f(x)}\,\xi(x)\,\text{d}x=\int_{\mathfrak X} \dfrac{\alpha(x)}{f(x)}\dfrac{f(x)}{c}\,\text{d}x=\int_{\mathfrak X} \dfrac{\alpha(x)}{c}\,\text{d}x=\frac{1}{c}$$Therefore, the estimate$$\frac{1}{n}\sum_{i=1}^n \dfrac{\alpha(x_i)}{f(x_i)}\qquad x_i\sim\xi(x)$$is an unbiased and convergent estimator of $1/c$, whatever $\alpha(\cdot)$ is. The only caution in choosing this $\alpha(\cdot)$ density is to ensure that the estimator has a finite variance, for otherwise the outcome is completely untrustworthy.

Answer 2

C is the integral of $f(x)$, if you divide $f(x)$ by its integral, the new $f(x)$ will be normalized as a PDF. You can obtain c by sampling uniformly in the support of $f(x)$. If U are the elements of this uniform sampling, c is = mean of $f(u)$ * support.

In the article they compute the integral analytically, so they don't need MC. Here I am showing a way to compute the integral by MC.

Here is an example. The support here is from 0 to 1. f is not normalized, so we obtain c.

f=function(x){2.75*dbeta(x,2,5)}
U=runif(10^7)
c=mean(f(U))*1
c
[1] 2.749145