3

I'm working on something about Least square estimation for Weibull distribution. After reviewing this paper, I'm pretty confused by the derivation of least square method.

In this paper, the author rank the data before calculating the least square of the "linear equation $(Y_i = aX_i+b)$" , why is that?

My second question is: why does the author use $i/(n+1)$ as the estimated value of cdf $F(x_{(i)})$? Is there any mathematical explanation on this problem?

Looking forward to any feedback and suggestions.

Chill2Macht
  • 5,639
  • 4
  • 25
  • 51
FantasticAI
  • 417
  • 1
  • 4
  • 12
  • 5
    Can you post more details from the paper? Low probability anybody here will read the paper for you! Any reason you cannot use the usual maximum likelihood estimators? – kjetil b halvorsen Apr 21 '17 at 17:43
  • 2
    I illustrate the four methods in [another post](https://stats.stackexchange.com/a/60530/21054) if your interested. – COOLSerdash Apr 21 '17 at 18:22

1 Answers1

4

I'll start with your second question first. Often, it is simpler to use $i/(n+1)$ (or $(i-1/2)/n$) instead of $i/n$ to avoid boundary issues. Let's say you've sorted your data so $Z_1,\ldots, Z_n$ refers to the smallest up to the biggest values in the data. Then, $Z_i$ is a pretty good estimate of the $(i/n)$th quantile of the distribution. But $i/n$ and $i/(n+1)$ are really close values and typically the true quantiles of $i/n$ and $i/(n+1)$ are also really close. Thus, we can argue that $Z_i$ is a pretty good estimate of the $(i/(n+1))$th quantile of the distribution. So, why does all this matter? Consider the biggest value in the data set, $Z_n$. Would this ever be a good estimate of quantile $n/n=1$? Of course not. For a Weibull distribution, quantile $1$ is always equal to $\infty$ yet $Z_n$ is always finite. That said, $Z_n$ is still a good approximation of quantile $n/(n+1)$.

As for your primary question, "why estimate by least squares?", you simply need to understand a simple fact of Weibull distributions. The Weibull CDF is given by $$F(x) = 1-\exp(-(x/\delta)^c)$$ If we rearrange this expression, we get $$\ln(x) = \dfrac 1 c \ln⁡(-\ln⁡(1-F(x))) + \ln⁡(\delta)$$ Now, if we define $Y_i = \ln(Z_i)$ and $X_i=\ln⁡(-\ln⁡(1-[i/(n+1)]))$, then we would expect to get the relationship, $Y_i = a X_i + b$, where $a=1/c$ and $b=\ln(\delta)$. All you have to do then is run a least squares regression of $X$ vs $Y$ and convert the parameter estimates into $c$ and $\delta$.

jjet
  • 1,187
  • 7
  • 12
  • +1 But is there any rationale behind treating $\ln Z_i$ as the dependent variable and $\ln(-\ln(1-i/(n+1)))$ as the independent variable in the least squares fit? The paper referred to by the OP does it the other way around. – Jarle Tufto Apr 21 '17 at 18:53
  • I didn't read the paper except to find the notation. However, I don't think it matters which variable is the regressor as you can still derive the parameters from the coefficients either way. The usual linear regression assumptions (independence, normality, etc.) don't hold either way you frame the problem. But my guess is the procedure will produce good estimates regardless. – jjet Apr 21 '17 at 18:57
  • @jjet Thank you for the answer. Another question is why we need to sort the data? or say why we use order statistics of X instead of the raw data? – FantasticAI Apr 21 '17 at 22:51
  • Well, when you do the regression of X on Y, you need the values to be matched up. The smallest X ($X_1$) corresponds to quantile $1/(n+1)$ and so forth. You could avoid sorting as long as you match up the quantiles appropriately. – jjet Apr 21 '17 at 23:29
  • Not sure if I totally understand what you're saying. But here's what you could write. First, define $X_(i)$ to be the ith smallest value in the data. Then, $F(X_(i))$ is a statistic and that statistic is usually a good estimator of $i/(n+1)$. – jjet Apr 28 '17 at 00:54