0

I have a set of percentiles the 10-th, 50-th and 90-th. Furthermore I have the mean value. I am trying to reconstruct the underlying distribution. The question is similar to

Estimating a distribution based on three percentiles

except I have a mean value as well.

My present approach was to fit a lognormal distribution based on the 50th percentile in which the variance was an unknown. Subsequently I determined the variance_1 based on the 10th percentile keeping the mean as found from the 50th. I did the same for the 90th percentile giving me variance_2. Then I searched for the variance somewhere between variance_1 and variance_2 which gave me the mean I was given in the first place.

10-th percentile: 0.004
50-th percentile: 0.007
90-th percentile: 0.02
arithmetic mean : 0.009

I came up with a log-normal (\mu = log(0.007), \sigma = 0.70825) distribution

What do you think of the approach, and how would you tackle this problem?

Community
  • 1
Willem
  • 101
  • 2
  • 2
    I would just like to remind you of the cautionary notes included in [my answer to the other question](http://stats.stackexchange.com/posts/6038/revisions), because they apply here, too. How *I* would tackle your problem would depend strongly on *why* you are fitting a distribution to these values and on the size of the dataset they are estimated from. Regardless, neither *ad hoc* methods (such as yours) nor "standard" methods such as a least-squares fit are truly appropriate due to the correlations among these statistics and the different levels of error that occur in measuring them. – whuber Jun 26 '14 at 03:48
  • Thanks @whuber. I realise that this is not a lot of data to make strong conclusions about the underlying data distribution. – Willem Jun 26 '14 at 04:10
  • 1
    It could be a huge amount of *data*, willem, as summarized with four statistics. So the amount of data isn't the issue. The keys to focus on in explaining your problem are (1) what is the purpose in fitting a distribution to these statistics and (2) what is the basis for assuming a lognormal distribution? – whuber Jun 26 '14 at 14:00
  • Are these statistics (the quantiles and mean) based on the usual samples estimates of quantiles and mean, or were they inferred from full data and estimation of lognormal parameters? What sort of sample sizes are they based on? – Glen_b Jun 26 '14 at 22:36
  • 1
    If we had lognormal data, and those quantiles were *sample* quantiles, and I only had the three quantiles, I'd use $k_n \log(x_{0.9}/x_{0.1})$ to estimate $\sigma$ (where $k_n$ is a scaling constant that will change with $n$ but in large samples will approach $1/2.56$), and I'd estimate $\mu$ by $\frac{1-a_n}{2} \log(x_{0.1})+ a_n \log(x_{0.5})+ \frac{1-a_n}{2} \log(x_{0.9})$, where the best choice of $a_n$ varies with $n$, but $a_n=0.5$ is very close to the optimum for both large and small $n$ (so I'd probably stick with that). Adding the mean (assuming it's a sample mean) complicates it – Glen_b Jun 28 '14 at 10:49
  • Its sediment data, and well graded mixtures often follow a lognormal distribution, so thats an empircal assumption you could choose. – Willem Jun 28 '14 at 12:18

1 Answers1

2

if you know the true mean and percentiles, then the fitting approach can be helpful and done by optimizers, but usually you only have estimates for mean and percentiles, like from a data set of 100 points. If you apply now an optimization for the fit the result is highly questionable. For instance a fit to moments should be more stable, best is a fit by MLE. Most approaches need first to make an assumption on the pdf (like data is lognormal), then finding the parameters. Or you may use a very flexible multi-parameter distribution which often ends up that the parameter estimation is not very stable. A good starting point is taking the empirical CDF as model + plus some smoothing (e.g. by kernal densitiy functions). This would be a many-parameter "model". Using less parameters give usually more systematic errors, but more stable parameters & estimations. So a compromize is needed, which e.g. depends on how many data points you have.

Bye Stephan

user32038
  • 409
  • 3
  • 10