0

This is a question concerning the width of the bar in a histogram. Let's say we have frequency distribution like this:

enter image description here

As far as I've learned when you define size of the bar for example 0.5, for each integer value you cluster all the points from the interval integer-0.5 and integer+0.5. So, for example if we draw bar around the integer 50 then it represents all the items from 49,8 + all the items from 50 + all the items in aggregate, that is 20+4+3=27 items in that subset. It doesn't necessarily have to be an integer value, but let's simplify it for this purposes!

Question: By taking a random observation from this distribution what is the probability that it will be in range >=50?

The way it's usually pictured is to sum all (elements contained in) bars in histogram from 50 till the end. If that is the case and thing I've previously described is true, then we should not inculde the subset from the 49,5 to get the right probability?

Maybe the error including that small subset is negligible if the size of the population is high, but still what if the size of the population is small?

Or, it is the other way around: To make a histogram, a size of a bar must be such that there is constant frequency around each number (so that you don't include anything around!).

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
Krushe
  • 41
  • 2

1 Answers1

1

Histograms are not really meant for such problems. They are nonparametric estimators for the probability density functions. For estimating probabilities, you would use empirical probability distribution function, i.e. just count the number of times $X \ge 50$, and divide by total size of your sample. By doing this you do not need to bother about bin size, because there is none. Using histograms in here would make the result depend on the bin size.

If the only thing that you are given is the histogram and you need to calculate the probability from it, then you can use a number of possible approaches, like rounding the values to the upper bounds of the bins, to the lower bounds, or using interpolation (linear, something smoother etc.), but in each case those are just workarounds that don't give you guarantees for correctness of the result, just a rough approximation. For example, this is that scipy.stats.rv_histogram class does:

class rv_histogram(rv_continuous):

    def __init__(self, histogram, *args, **kwargs):
        self._histogram = histogram
        self._hpdf = np.asarray(histogram[0])
        self._hbins = np.asarray(histogram[1])
        self._hbin_widths = self._hbins[1:] - self._hbins[:-1]
        self._hpdf = self._hpdf / float(np.sum(self._hpdf * self._hbin_widths))
        self._hcdf = np.cumsum(self._hpdf * self._hbin_widths)

    def _cdf(self, x):
        return np.interp(x, self._hbins, self._hcdf)

(The code was simplified by me. I omitted some of the details that are not important for this problem.)

Tim
  • 108,699
  • 20
  • 212
  • 390