1

I am trying to get the values of the 25th and 75th quantile of the population based on two values that summarizes the samples:

  • median value
  • 90 percent margin of error

I don't have any other information including the sample size, standard error, etc.. And I think it 's safe to assume the samples were drawn from a normal distribution.

The 90 percent margin of error in the original document is described as follows:

The degree of uncertainty for an estimate arising from sampling variability is represented through the use of a margin of error. The value shown here is the 90 percent margin of error. The margin of error can be interpreted roughly as providing a 90 percent probability that the interval defined by the estimate minus the margin of error and the estimate plus the margin of error (the lower and upper confidence bounds) contains the true value.

Edit: added the description of the margin of error to clarify the question.

Xiaoyu Lu
  • 209
  • 2
  • 12

2 Answers2

1

Assume a sample size of 200, with mean (mu) = 20, and standard deviation (sigma) = 10.

import numpy as np

mu, sigma = 20, 10 # mean and standard deviation
s = np.random.normal(mu, sigma, 200)

np.quantile(s, 0.25)
np.quantile(s, 0.75)

I'm using Python for this example, but you can see that we are:

1) generating an array of 200 normally distributed random numbers

2) Obtaining the 25th and 75th quantile.

>>> np.quantile(s, 0.25)
11.700325588242732
>>> np.quantile(s, 0.75)
26.11671871467393

Now, when you say "90% margin of error", I am assuming you mean a 90% "confidence interval". In this case, your margin of error is 10%.

Using the scipy library (also from Python), we can obtain a 90% confidence interval as follows:

from scipy import stats
stats.norm.interval(0.90, loc=mu, scale=s)

More detail can be found on the above here.

You can now see that we generate an array where the values would fall within the 90% confidence interval:

>>> stats.norm.interval(0.90, loc=mu, scale=s)
(array([-20.1017426 , -50.41395259, -15.74140484, -34.9162548 ,
       -14.55505407, -26.20186343,  -8.38349335, -28.15329328,
............
         0.3405667 ,  14.1913693 , -44.18605464, -18.30478346]), 
array([60.1017426 , 90.41395259, 55.74140484, 74.9162548 , 54.55505407,
       66.20186343, 48.38349335, 68.15329328, 42.42820445, 70.17147704,
............
       55.23983044, 41.10373296, 51.30638793, 57.20990033, 47.99641712]))

The above is obviously dependent on which software you are using and what dataset you are working with, but hopefully you might find these guidelines useful.

Michael Grogan
  • 1,435
  • 1
  • 8
  • 11
  • Michael, Thanks a lot for the response. In my case, the sigma is unknown. Also, the `stats.norm.interval` function seems to produce the lower/upper bounds instead of pseudo data, so I cannot use the `np.quantile` based on the output. Am I missing out something? Thanks. – Xiaoyu Lu Sep 28 '18 at 21:16
  • When you say "the population" in your question, do you actually have a dataset that you are using? If you have data, then you should be able to calculate the standard deviation, i.e. sigma. – Michael Grogan Sep 28 '18 at 22:26
  • I do not have the dataset. I only have the two values (median, margin of error), and intend to generate the quantiles on top of that. I have added more description of margin of error in the original post. – Xiaoyu Lu Sep 28 '18 at 22:52
  • "90 percent probability that the interval defined by the estimate minus the margin of error and the estimate plus the margin of error (the lower and upper confidence bounds) contains the true value". 90 percent does not refer to a "margin of error". It refers to a confidence interval. Given that you only have the median and the confidence interval, then you would need to make assumptions regarding your data, e.g. random number generation as in the example. And, as EdM very correctly stated, you would need to know the size of the sample for your confidence interval to have any meaning. – Michael Grogan Sep 29 '18 at 13:00
1

The use of the phrase "upper and lower confidence bounds" to describe the "90% margin of error" in your cited source suggests that it is referring to 90% confidence limits. The explanation in the citation, however, is incorrect; see this page for extensive discussion of what frequentist confidence intervals and Bayesian credible intervals actually represent.

If the data are from a normal distribution then the median provides an estimate of the population mean. The confidence interval, however, depends on the size of the sample. If you only had 2 cases in your sample the confidence interval around the mean would be very wide. If you had 2000 cases in your sample then it would be quite narrow. So the "90% margin of error" doesn't provide information about the width of the distribution in the population unless you also know the size of the sample.

Thus the answer by Michael Grogan describes how you can proceed with your analysis. You need to know the sample size, too.

EdM
  • 57,766
  • 7
  • 66
  • 187
  • 1
    Thank you for the information. It is sad that the quotation was actually from the American Community Survey household income dataset, and one would hope the statisticians in the Census Bureau could do a better job at providing more accurate descriptions. With that being said, the data I was looking at did not provide a sample size, but I can assume the sample size being large since it's aggregated using zipcodes. I think I will try some other approaches such as Monte Carlo to generate pseudo data. – Xiaoyu Lu Sep 29 '18 at 16:08
  • @XiaoyuLu : Sample size information is available on-line for the ACS; see [this page](https://www.census.gov/acs/www/methodology/sample-size-and-data-quality/sample-size/index.php) for nationwide values, for example. – EdM Sep 30 '18 at 04:10
  • Thanks again. Correct me if I am wrong, it seems to me that the ACS seems to have sample size information only on State level and National level. I am looking at zip-code level data, and I failed to find sample size information. Was I missing something? – Xiaoyu Lu Oct 01 '18 at 14:07
  • @XiaoyuLu you might consider using the detailed [Public Use Microdata Sample](https://www.census.gov/programs-surveys/acs/technical-documentation/pums/about.html) rather than summary files; for confidentiality no ZIP-code data but rather data for defined areas with at least 100,000 people. There might be metadata somewhere that contain the number of total cases per ZIP code; try the "Contact us" link on that page. For rough estimates you could find population by ZIP code and assume the 1% intended 1-year sampling rate. The linked page also contains links to getting point estimates and errors. – EdM Oct 01 '18 at 15:03
  • @XiaoyuLu the Census Bureau does define confidence intervals correctly in the [ACS accuracy documentation](https://www2.census.gov/programs-surveys/acs/tech_docs/accuracy/ACS_Accuracy_of_Data_2017.pdf?#) on pages 21-22. You might even find a way to use the margins of error reported for ACS 5-year population by ZIP code to get a useful estimate of the actual number of sampled cases. The document also has detailed information on how the error estimates are obtained. – EdM Oct 01 '18 at 15:37