Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [efficiency]

A measure of the quality of a statistical estimator.

Questions with the efficiency tag typically concern how to model a parameter or parameters more accurately or (almost equivalently) how to minimize the amount of data needed to achieve a given level of estimation accuracy.

132 questions
58
votes
8 answers

Examples where method of moments can beat maximum likelihood in small samples?

Maximum likelihood estimators (MLE) are asymptotically efficient; we see the practical upshot in that they often do better than method of moments (MoM) estimates (when they differ), even at small sample sizes Here 'better than' means in the sense…
Glen_b
  • 257,508
  • 32
  • 553
  • 939
22
votes
11 answers

Does machine learning really need data-efficient algorithms?

Deep learning methods are often said to be very data-inefficient, requiring 100-1000 examples per class, where a human needs 1-2 to reach comparable classification accuracy. However, modern datasets are huge (or can be made huge), which begs the…
MWB
  • 1,143
  • 9
  • 18
20
votes
2 answers

Why $\sqrt{n}$ in the definition of asymptotic normality?

A sequence of estimators $U_n$ for a parameter $\theta$ is asymptotically normal if $\sqrt{n}(U_n - \theta) \to N(0,v)$. (source) We then call $v$ the asymptotic variance of $U_n$. If this variance is equal to the Cramer-Rao bound, we say the…
clueless
  • 303
  • 2
  • 6
19
votes
2 answers

For what (symmetric) distributions is sample mean a more efficient estimator than sample median?

I have labored under the belief that the sample median is more robust measure of central tendency than the sample mean, since it ignores outliers. I was therefore surprised to learn (in the answer to another question) that for samples drawn from a…
Josh Brown Kramer
  • 597
  • 3
  • 12
14
votes
3 answers

Why is the asymptotic relative efficiency of the Wilcoxon test $3/\pi$ compared to Student's t-test for normally distributed data?

It is well-known that the asymptotic relative efficiency (ARE) of the Wilcoxon signed rank test is $\frac{3}{\pi} \approx 0.955$ compared to Student's t-test, if the data are drawn from a normally distributed population. This is true for both the…
Silverfish
  • 20,678
  • 23
  • 92
  • 180
9
votes
1 answer

Relative efficiency of Wilcoxon signed rank in small samples

I have seen in published literature (and posted on here) that the asymptotic relative efficiency of the Wilcoxon signed rank test is at least 0.864 when compared to the t test. I have also heard that this only applies to large samples, although some…
Jimj
  • 1,053
  • 1
  • 14
  • 25
9
votes
1 answer

When can't Cramer-Rao lower bound be reached?

The Cramer-Rao lower bound (CRLB) gives the minimum variance of an unbiased estimator. One sentence in the wiki page says "However, in some cases, no unbiased technique exists which achieves the bound. This may occur either if for any unbiased…
Tony B
  • 210
  • 1
  • 5
9
votes
2 answers

Is OLS Asymptotically Efficient Under Heteroscedasticity

I know that OLS is unbiased but not efficient under heteroscedasticity in a linear regression setting. In Wikipedia http://en.wikipedia.org/wiki/Minimum_mean_square_error The MMSE estimator is asymptotically unbiased and it converges in…
Cagdas Ozgenc
  • 3,716
  • 2
  • 29
  • 55
8
votes
3 answers

The concept of efficiency

I have some problems in understanding the concept of efficiency as related to an estimator. My sources (Mukhopadhyay, 2000 and Casella, Berger, 2002) do not treat this argument as I expected since they analyse only the concept of asymptotic…
PhDing
  • 2,470
  • 6
  • 32
  • 57
7
votes
0 answers

Why does MLE tend to normal distribution

We have $X_1,\dots, X_n$ are iid (the distribution can be of any type, e.g. Bernoulli (p), normal ($\mu, \sigma^2$), Poisson ($\lambda$). If we use MLE $\hat \theta$ to estimate any parameter $\theta$ of the distribution,, then, as is said in…
Charlie Chang
  • 233
  • 1
  • 6
7
votes
1 answer

Efficient Estimator from Insufficient Statistic

Suppose that I have a statistic $T(X)$, and I know for sure that it is not sufficient to estimate a parameter $\theta$. Is it still possible to have an estimator $\hat\theta(T(X))$ that is efficient (under convex loss), or is there a theorem…
Cagdas Ozgenc
  • 3,716
  • 2
  • 29
  • 55
7
votes
1 answer

Why efficiency matters?

Suppose we are trying to estimate the quantity $\theta$ and we have that the estimator $\hat\theta_n$. Suppose it is efficient, i.e. is variance is the smallest among certain class of other possible estimators of $\theta$, say that this class is a…
mpiktas
  • 33,140
  • 5
  • 82
  • 138
7
votes
1 answer

If $\operatorname{Var}\left(\epsilon_i\right) = h\left(X\right) \neq \sigma^2$, what can we know about $\operatorname{Var}\left(\hat{\beta}\right)$?

This question uses the derivations found here. The short version Consider a regression model. If the error variance is a known function of the data (rather than a constant), under what conditions can we draw conclusions about the OLS estimates? The…
shadowtalker
  • 11,395
  • 3
  • 49
  • 109
7
votes
1 answer

Is $X_{(1)} + X_{(n)}$ a good estimator for $\theta$?

Problem 8.7 From Van der Vaart's Asymptotic Statistics: Given a sample of size $n$ from the uniform distribution on $[0,\theta]$, the maximum $X_{(n)}$ of the observations is biased downwards. Because $\text{E}[\theta-X_{(n)}] = \text{E}[X_{(1)}]$,…
bdeonovic
  • 8,507
  • 1
  • 24
  • 49
6
votes
2 answers

Is MLE more efficient than Moment method?

I have got some small data sets (about 8 to 11 data points for each set), following Normal distribution. I would like to find out the 95% confidence interval of the 0.005 and 0.995 percentile of each set. Firstly, moment estimation method is…
Shusheng
  • 106
  • 5
1
2 3
…
8 9