Questions tagged [minimum-variance]
25 questions
9
votes
1 answer
Optimal importance sampling with ratio estimator
We want to approximate the following expectation:
$$\mathbb{E}[h(x)] = \int h(x)\pi(x) dx$$
Where $h(x)$ is an arbitrary function and $\pi(x)$ is a distribution, also for simplicity, let's assume that we actually know the normalizing constant for…
fairidox
- 1,188
- 5
- 16
8
votes
0 answers
Methods of Proving that a UMVUE does not exist?
Are there efficient methods of showing when a UMVUE does not exist? I can think of the trivial case when no unbiased estimators exist at all. But that's not really interesting.
I feel like this would be difficult because there could be infinitely…
TheChosenOne
- 81
- 2
4
votes
2 answers
How is this minimum variance worked out for this importance sampling estimator?
I was stuck with the function 17.13 in the open source book deep learning on page 590.
For short, the question is that,
For the importance sampling estimator:
$$\hat s_q = \frac{1}{n}\sum_{i=1, x^{i}\sim…
Lerner Zhang
- 5,017
- 1
- 31
- 52
4
votes
1 answer
Minimum-variance unbiased linear estimator
Suppose that it is known that the mean of RV $X_i$ is $\mu_i\theta$, (i = 1, 2,..., n), where $\mu_i$ are known constants, whereas $\theta$ is unknown. Let $\Sigma$ be the variance matrix of the random vector $X = \left[X_1, X_2,\ldots,…
Dony
- 131
- 2
3
votes
1 answer
Error in Derivation for Control Variate Variance?
I'm trying to derive the variance for a control variate estimator, but I seem to be missing a term that allows me to end up with the covariance in the final answer.
Let $f(x)$ be my function and let $h(x)$ be my control variate with $x \sim p(x)$.…
Rylan Schaeffer
- 689
- 3
- 10
3
votes
1 answer
Good parameter estimates vs good computed moment estimates
Suppose I have a distribution from a known parametric family f(x; θ). I have a sample from that distribution. From the sample, I estimate values for the parameters. Suppose I have estimators that I know to have some desirable property. For instance,…
andrewH
- 2,587
- 14
- 27
3
votes
1 answer
Are MVUEs and MLEs always functions of a minimal sufficient statistic?
Is it the case that both minimum variance unbiased estimators (MVUEs) and maximum likelihood estimators (MLEs) are always functions of a minimal sufficient statistic?
If so, how do we know? If not, what are some exceptions or what are some ways that…
Bratt Swan
- 653
- 2
- 7
- 12
3
votes
0 answers
A Proof of Tukey's Inequality
Suppose that $W_1,W_2,...,W_n$ are uncorrelated unbiased estimators of a parameter $\theta$.
Consider $W=\sum_{i=1}^na_iW_i$ such that $E(W)=\theta$ and $Var(W_i)=\sigma^2_i$, where the $a_i$'s are constants. So it is trivial to see that…
Landon Carter
- 1,295
- 11
- 21
2
votes
1 answer
How does one can guarantee that any unbiased estimator is MVUE due to it containing a minimal sufficient statistic?
Sufficiency is okay. But I don't really get it why the fact guarantees it has minimal variance? Can anyone explain to me somewhat intuitively?
Jason Park
- 23
- 5
1
vote
1 answer
Experimental Design: Choose Data Points to Minimize Quadratic Term Variance in Multiple Regression
$\newcommand{\eps}{\varepsilon}\newcommand{\szdp}[1]{\!\left(#1\right)}
\newcommand{\szdb}[1]{\!\left[#1\right]}$
Problem Statement: Suppose that you wish to fit a model
$$Y=\beta_0+\beta_1x+\beta_2x^2+\eps$$
to a set of $n$ data points. If the $n$…
Adrian Keister
- 3,664
- 5
- 18
- 35
1
vote
0 answers
How to find an minimum variance unbiased estimator for an integer parameter?
Consider multiple observations $x[n]$ for an integer parameter $A$ under White Gaussian Noise $w[n]$:
$x[n]=A+w[n]; \quad$ $n=0,1,...,N−1$ with $w[n] \sim N(0,σ^2)$.
Is it possible to have an minimum variance unbiased estimator for the integer…
Thiruppathirajan
- 41
- 3
1
vote
1 answer
Rao-Blackwell for Minimum-Variance Unbiased Estimator
Let $X$ be an observation from a distribution with probability mass
function:$f(x;\theta) =
\left(\frac{\theta}{2}\right)^{|x|}(1-\theta)^{1-|x|}I_{\{-1,0,1\}}(x),
\, \theta \in (0,1).$ Use Rao-Blackwell theorem to find the
Minimum-Variance…
Andrew
- 133
- 3
1
vote
0 answers
Understanding Rao-Blackwell
From Casella and Berger:
Let $W$ be an unbiased estimator of $\tau(\theta)$ and let $T$ be a sufficient statistics for $\theta$. Define $\phi(T) = E[W|T]$. Then $E_{\theta}[ \phi(T)] = \tau(\theta)$ and $Var_{\theta} ( \phi(T)) \leq Var_{\theta}…
Marcel
- 1,200
- 6
- 24
1
vote
1 answer
Better to Minimize Absolute Error or Sum of Squared Error?
I have an Excel model which predicts the number of customers for a given month. The prediction depends on a churn rate. I have the absolute error (actual vs predicted), along with squared error and sum of square error.
My question is:
Would it…
alexsmith2
- 161
- 1
- 1
- 4
1
vote
1 answer
How to measure how "good" or accurate a probability distribution is? Entropy, variance or what?
How can one measure the accuracy of the probability distribution of, say, a physical magnitude?
I know one good candidate is the entropy, which measures the amount of information one has about the system---cf. Appendix A of Ref. [E. Jaynes, Physical…
Godoy
- 23
- 4