Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [blue]

Best Linear Unbiased Estimator

BLUE stands for Best Linear Unbiased Estimator. The phrase comes from the Gauss-markov theorem that the OLS estimator is BLUE when the standard linear regression assumptions (i.e., the errors are independent and identically distributed as a Gaussian with constant variance). The acronym can be broken down as follows:

  • An estimator is a specified calculation used on sample data to estimate a quantity (e.g., a population parameter--see also ).
  • "Unbiased" means that the expected value of the estimator equals the parameter being estimated (conceptually, the sampling distribution of the resulting estimates is centered on the true value--see also , and ).
  • "Linear" refers to linear in the parameters.
  • "Best" implies that the sampling distribution of the resulting estimates having the lowest variance of all possible unbiased linear estimators.
39 questions
76
votes
1 answer

Impractical question: is it possible to find the regression line using a ruler and compass?

The ancient greeks famously sought to construct geometrical relationships using only a ruler and a compass. Given a set of points in a two dimensional plane, is it possible to find the OLS line using only such instruments? This question has…
Pablo Derbez
  • 808
  • 3
  • 9
23
votes
1 answer

Why do the estimated values from a Best Linear Unbiased Predictor (BLUP) differ from a Best Linear Unbiased Estimator (BLUE)?

I understand that the difference between them is related to whether the grouping variable in the model is estimated as a fixed or random effect, but it's not clear to me why they are not the same (if they are not the same). I am specifically…
Jeremy Miles
  • 13,917
  • 6
  • 30
  • 64
15
votes
1 answer

Other unbiased estimators than the BLUE (OLS solution) for linear models

For a linear model the OLS solution provides the best linear unbiased estimator for the parameters. Of course we can trade in a bias for lower variance, e.g. ridge regression. But my question is regarding having no bias. Are there any other…
Gumeo
  • 3,551
  • 1
  • 21
  • 31
11
votes
1 answer

Why is bias equal to zero for OLS estimator with respect to linear regression?

I understand the concept of bias-variance tradeoff. Bias based on my understanding, represents the error because of using a simple classifer(eg: linear) to capture a complex non-linear decision boundary. So I expected OLS estimator to have high bias…
GeorgeOfTheRF
  • 5,063
  • 14
  • 42
  • 51
9
votes
2 answers

Gauss-Markov theorem: BLUE and OLS

I'm reading up on the Guass-Markov theorem on wikipedia, and I was hoping somebody could help me figure out the main point of the theorem. We assume a linear model, in matrix form, is given by: $$ y = X\beta +\eta $$ and we're looking for the BLUE,…
Patrick
  • 761
  • 4
  • 14
8
votes
0 answers

Is the OLS estimator the UMVUE (assuming Normality)?

Suppose $$ \mathbf{y} = \mathbf{X} \mathbf{b} + \mathbf{e} \, , \\ \mathbf{e} \sim \mathcal{N}(0,\mathbf{I}_P) \, . $$ We know that $\mathbf{\hat{b}} = (\mathbf{X}^T \mathbf{X})^{-1} \mathbf{X}^T \mathbf{y}$ is the BLUE. Is it also the UMVUE? I can…
Patrick
  • 761
  • 4
  • 14
8
votes
2 answers

What are the properties of MLE that make it more desirable than OLS?

This question seems fundamental enough that I'm convinced it has been answered here somewhere, but I haven't found it. I understand that if the dependent variable in a regression is normally distributed, maximum likelihood and ordinary least squares…
Michael Webb
  • 1,936
  • 10
  • 21
7
votes
3 answers

Proving Linear Estimator (beta) is BLUE?

In the book Statistical Inference pg 570 of pdf, There's a derivation on how a linear estimator can be proven to be BLUE. I got all the way up to 11.3.18 and then the next part stuck me. After finding the $d_i$s that satisfy the condition: $\Sigma…
Kevin Pei
  • 749
  • 2
  • 9
  • 13
6
votes
4 answers

What is the benefit of regression with student-t residuals over OLS regression?

Sometimes I see advice to fit regressions with student-t residuals rather than using OLS (which is equivalent to assuming normally distributed residuals) if the distribution of the residuals is heavy-tailed. However, since the OLS estimator is BLUE…
frelk
  • 1,117
  • 1
  • 8
  • 19
5
votes
0 answers

Generalized Least Square When Disturbance Covariance Matrix Is Rank Deficient

I cannot find any general results on the following Generalized Least Square (GLS) problem. Let $Y = X\beta + E$, where $X$ is deterministic and of full column rank $k$, and $E$ is of zero mean, with a $n$-by-$n$ covariance matrix $V$ with rank $r <…
Shang Zhang
  • 431
  • 1
  • 6
5
votes
2 answers

Prove that the OLS estimator of the intercept is BLUE

Consider the simple linear regression model $$y_i = \alpha + \beta x_i + u_i$$ with classic Gauss-Markov assumptions. In proving that $\hat{\beta}$, the OLS estimator for $\beta$, is the best linear unbiased estimator, one approach is to define an…
greggs
  • 190
  • 8
5
votes
2 answers

Why doesn't the Cramer-Rao lower bound apply?

Let $X_1, X_2, \dots, X_n$ be a sample of i.i.d. random variables, with density $$f_\theta=\frac{2}{3\theta}\left(1-\frac{x}{3\theta}\right) $$ for $0 < x < 3\theta$. And $f_\theta=0$ if $ x < 0$ or $ x>3\theta$ Let $\hat{\theta}=\overline{X}$ be an…
Applied mathematician
  • 391
  • 1
  • 5
  • 13
4
votes
1 answer

Proof for "Least squares estimator is BLUE"

I checked all the books and on-line materials I could find for the proof, but found all of them have a derivation problem, which I cannot understand. To prove the least squares estimator is the $BLUE$ for the linear model $y = X*\beta + v$, one…
Alan
  • 51
  • 1
  • 2
4
votes
0 answers

In Ordinary Least Square (OLS) estimation: is the slope actually an "Inverse-variance weighting" estimator?

I am suspecting the answer is yes, but I'd appreciate help in proving it (even though we know that the estimator is BLUE, so it should probably hold). For context: An Inverse-variance weighting is when we have a bunch of estimators for some…
Tal Galili
  • 19,935
  • 32
  • 133
  • 195
4
votes
2 answers

What's the difference between "Optimal linear predictor" and "best unbiased linear estimator"?

Greene (econometric analysis 7th ed. p 53) states that OLS is the "optimal linear predictor": Then on the next page, he states that OLS is also the BLUE estimator (Gauss-Markov Theorem): I understand the proof of the Gauss Markov theorem.…
user56834
  • 2,157
  • 13
  • 35
1
2 3