Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [rao-blackwell]

The Rao-Blackwell Theorem is a result which makes possible to better an estimator by conditioning on a sufficient statistic, preserving the expectation of the estimator.

Wikipedia has an article https://en.wikipedia.org/wiki/Rao%E2%80%93Blackwell_theorem with further references.

53 questions
11
votes
1 answer

Rao-Blackwellization of sequential Monte Carlo filters

In the seminal paper "Rao-Blackwellised Particle Filtering for Dynamic Bayesian Networks" by A. Doucet et. al. a sequential monte carlo filter (particle filter) is proposed, which makes use of a linear substructure $x^L_k$ in a markov process $x_k =…
Jakob
  • 281
  • 1
  • 8
10
votes
2 answers

Why does the Rao-Blackwell Theorem require $\Bbb E(\hat{\theta}^2) < \infty$?

The Rao-Blackwell Theorem states Let $\hat{\theta}$ be an estimator of $\theta$ with $\Bbb E (\hat{\theta}^2) < \infty$ for all $\theta$. Suppose that $T$ is sufficient for $\theta$, and let $\theta ^ * = \Bbb E (\hat{\theta}|T)$ Then for all…
Stan Shunpike
  • 3,623
  • 2
  • 27
  • 36
10
votes
2 answers

Find the joint distribution of $X_1$ and $\sum_{i=1}^n X_i$

This question is from Robert Hogg's Introduction to Mathematical Statistics 6th version question 7.6.7. The problem is : Let a random sample of size $n$ be taken from a distribution with the pdf…
Deep North
  • 4,527
  • 2
  • 18
  • 38
9
votes
0 answers

Rao-Blackwellization in variational inference

The Black box VI paper introduces Rao-Blackwellization as a method to reduce the variance of the gradient estimator using score function, in section 3.1. However I don't quite get the basic idea behind those formulas, please give me some hint and…
avocado
  • 3,045
  • 5
  • 32
  • 45
9
votes
2 answers

Rao-Blackwellization of Gibbs Sampler

I am currently estimating a stochastic volatility model with Markov Chain Monte Carlo methods. Thereby, I am implementing Gibbs and Metropolis sampling methods.Assuming I take the mean of the posterior distribution rather than a random sample from…
mscnvrsy
  • 515
  • 6
  • 15
8
votes
2 answers

What is the necessary condition for a unbiased estimator to be UMVUE?

According to the Rao-Blackwell theorem, if statistic $T$ is a sufficient and complete for $\theta$, and $E(T)=\theta$, then $T$ is a uniformly minimum-variance unbiased estimator (UMVUE). I am wondering how to justify that an unbiased estimator is…
Alex Brown
  • 81
  • 1
  • 3
7
votes
2 answers

Using all Metropolis-Hastings proposals to estimate an integral

Suppose we run the Metropolis-Hastings with target distribution $\mu$ to compute the integral $\int f\:{\rm d}\mu$. Usually, we use the estimator $$A_n:=\frac1n\sum_{i=0}^{n-1}f(X_i).$$ However, instead of adding up the $f(X_i)$, I've seen the…
0xbadf00d
  • 539
  • 3
  • 8
5
votes
1 answer

Trying to make sense of claims regarding Rao-Blackwell and Lehmann-Scheffé for sufficient/complete statistics

I am currently trying to learn the two related concepts of the Rao-Blackwell theorem and the Lehmann-Scheffé theorem. Assume we have the random sample $X_1, \dots, X_n$ with mean $\mu$ and variance $\sigma^2 < \infty$. We have that $E[S^2] =…
The Pointer
  • 1,064
  • 13
  • 35
5
votes
2 answers

Finding UMVUE for a function of a Bernoulli parameter

Given $m$ i.i.d. Bernoulli( $\theta$ ) r.v.s $X_{1}, X_{2}, \ldots, X_{m},$ I'm interested in finding the UMVUE of $(1-\theta)^{1/k}$, when $k$ is a positive integer. . I know $\sum X_{i}$ is a sufficient statistic by the Factorization Theorem, but…
wanderer
  • 211
  • 1
  • 8
5
votes
1 answer

Finding UMVUE of Bernoulli random variables

Given i.i.d. Bernoulli$\left(\theta\right)$ r.v.s $X_1, X_2, ...,X_n$, I'm asked to solve for the UMVUE of $\left(1−\theta\right)^2$ for the case $n=4$. I think that $\sum X_i$ is my sufficient statistic (and I think completely sufficient) from…
Guest
  • 73
  • 1
  • 4
4
votes
0 answers

Rao-Blackwellization in Black Box VI

In the paper, "Black Box Variational Inference," by Ranganath et al. (2013), the authors derive a Rao-Blackwellized estimator of the gradient of the evidence lower bound with respect to a single variational parameter $\lambda_i$. Notation: $q(z |…
Ethan S
  • 41
  • 2
4
votes
1 answer

Relationship between completeness and sufficiency

hopefully this isn't a duplicate of another question (at least I didn't find one). Here is a question I have about completeness and sufficiency: Problem: Suppose $T(x)$ is complete sufficient for $\theta$ given data $x$. Show that if a minimal…
asdf
  • 364
  • 1
  • 10
4
votes
1 answer

Proof of Rao Blackwellization

I am reading this classic paper (Information and the Accuracy Attainable in the Estimation of Statistical Parameters) by CR Rao where he introduces the idea of minimizing the variance of an unbiased estimate using sufficient statistics. I have some…
honeybadger
  • 1,262
  • 8
  • 23
4
votes
0 answers

Minimum-variance unbiased estimator to estimate quantiles when the errors are normal distributed

What is the minimum-variance unbiased estimator to estimate quantiles when the errors are normal distributed? median When we wish to estimate the median, $\mu$, of a normal distributed variable then the sample mean (an efficient estimator of $\mu$)…
Sextus Empiricus
  • 43,080
  • 1
  • 72
  • 161
3
votes
1 answer

Rao–Blackwellization of Metropolis–Hastings

I am trying to achieve a Rao–Blackwellization of Metropolis–Hastings algorithm. In the paper by Robert et al. 2018, the following is given. \begin{align} ℑ=&\frac{1}{T}\sum_{t=1}^Th(\theta^{(t)})=\frac{1}{T}\sum_{i=1}^Th(ϑ_t)\sum_{t=1}^T…
boyaronur
  • 133
  • 5
1
2 3 4