Why bootstrapping?

Question

I understood that bootstrapping is a technique used to estimate statistics of a population. In bootstrapping we take many samples of chosen size, estimate statistics and obtain the mean of these statistics. This mean is representative of the whole population.

My doubt is that why take the samples in the first place. If you have the whole population with you, calculate the statistics on the whole for which you get 100% accurate statistics?

In bootstrapping as in other forms of inference one does not know that exact population (or have all the values in it). Otherwise, there would be no need for inference. In nonparametric bootstrapping, re-sampling is from from the _sample,_ not population. [In parametric bootstrapping, one has point estimates of parameters and to get CIs, re-samples from a population with the estimated parameters; so again one is not re-sampling from the original population.] — BruceET, Oct 29 '18 at 07:42
*Everything* we do in statistics is about using sample data to estimate the population parameters, if you have population data you don't need statistics, see https://stats.stackexchange.com/questions/2628/statistical-inference-when-the-sample-is-the-population — Tim, Oct 29 '18 at 07:54
vamsi is asking why sample from the sample? i would venture to say there is something in repeatedly subsampling the sample you get than just deriving summary statistics from that sample — jermdemo, Apr 10 '19 at 18:14

score 10 · Answer 1 · answered Oct 29 '18 at 06:35

Welcome to CV!

In bootstrapping, you repeatedly take samples with replacement from the original sample. The general idea behind this is that if you can estimate the uncertainty in your sample by asking the question: What if I didn't observe this observation, or that one, or if I observed this observation more than once?

You do this, say, $B = 1,000$ times, and end up with $1,000$ slightly different estimates of your statistic of interest. Depending on how strongly the calculated statistic is affected by this, the variance of your bootstrapped statistic will be larger.

In fact, it turns out that the standard deviation of the bootstrapped statistic can be a really good estimator of the standard error of your statistic.

And so, by simply randomly resampling our original sample with replacement, over and over, we have obtained an idea of how precise the estimate is, given that we only have a sample of the population.

Of course, if you can measure the entire population, then there is no point in bootstrapping.

score 2 · Answer 2 · edited Oct 22 '19 at 21:28

I understood that bootstrapping is a technique used to estimate statistics of a population.

It is a technique mainly used

to estimate the standard error of an estimator of a population parameter $\theta$ and/or
to derive confidence intervals for $\theta$

in situations where these figures are too difficult to derive by mathematical statistics.

In bootstrapping we take many samples of chosen size, estimate statistics and obtain the mean of these statistics. This mean is representative of the whole population.

It is indeed a resampling technique and works by sampling $n$ observations with replacement from the original $n$ observations. The key statement is that such resampled bootstrap sample is to the original sample as the original sample is to the population. Not the mean of the bootstrapped statistic is usually of interest, but rather the variation as it allows to find confidence intervals.

My doubt is that why take the samples in the first place. If you have the whole population with you, calculate the statistics on the whole for which you get 100% accurate statistics ?

Normally, we do not have access to the whole population. But if we have, indeed there is usually no need to do any inferential statistics (including the bootstrap).

Why bootstrapping?

2 Answers2