0

I'm trying to convince myself that bootstrapping fails when estimating extreme order statistics (and thus functions thereof). This is a classic shortcoming of the bootstrap laid out in some detail by Chernick (2007, 2011). The reasoning for this is that other methods have been proposed to get around the issue of extreme values, such as the $m$-out-of-$n$ bootstrap.

Consider bootstrapping sample range $R = X_{(n)} - X_{(1)}$ from a $N(5, 1.5)$ distribution from $n$ = 100 observations.

This is easily accomplished in R as follows.

set.seed(1991)

x <- rnorm(100, 5, 1.5) # simulated data
r <- max(x) - min(x) # true value = 10.02242

f <- function(x, i) {
   return(max(x[i]) - min(x[i])) 
}

library(boot)

y <- boot(x, f, R = 10000)

plot(y) # histogram and QQ plot

boot.ci(y) # confidence intervals
Intervals : 
Level      Normal              Basic         
95%   ( 8.97, 13.40 )   (10.02, 13.55 )  

Level     Percentile            BCa          
95%   ( 6.50, 10.02 )   ( 7.12, 10.02 )  
Calculations and Intervals on Original Scale
Some BCa intervals may be unstable

The true range is captured in two of the above intervals.

I also notice that the true/bootstrapped value is often the endpoint of the CI. I don't think this is a random occurrence, since R sometimes warns users of this, but I still consider the endpoint value falling within the interval.'

Here are the plots of the bootstrap distribution. This is what I'm getting at. Looking at these, it suggest something might be wrong -- and indeed this is the case.

The below plots look nothing like your typical bootstrap distribution when $n$ is large. Typically, when we bootstrap "nice" (smooth) statistics like the sample correlation say, the plots end up approximating a Gaussian distribution quite closely.
enter image description here

Besides checking whether or not bootstrap confidence intervals capture the true parameter value (and looking at the distribution plots), is there any other evidence suggesting that the traditional bootstrap fails in the case of estimating the sample distribution of extreme order statistics? I'm thinking that interval coverage could also be examined, but is this typically done (or is there enough proof in the intervals and plots themselves)?

Edit - In response to duplicate question Many responses to similar questions on CV are quite theoretical. While I appreciate and understand the theory behind the bootstrap, many researchers in other fields do not and would appreciate applied answers that use simulations/real examples in R.

compbiostats
  • 1,017
  • 9
  • 18
  • I've read that post. It's helpful, but I'd like to see an R-centric response rather than a theoretical one. I have provided an edit in my post to reflect this. – compbiostats Nov 28 '20 at 20:33
  • I don't know what you mean by an applied answer. Simulations don't disprove convergence even though with a large number of replications you can get an indication of a ack of convergence to the required distribution. – Michael R. Chernick Nov 28 '20 at 21:38
  • @MichaelR.Chernick Applied in the sense of using statistical software to carry out simulations or employ real data is all I mean by this. Your 2011 book employs R, but doesn't explicitly demonstrate bootstrap failure with a specific numerical example, through for instance the 'boot' R package. – compbiostats Nov 28 '20 at 22:15
  • I don't understand your question. It asks "is there anything else to look for..." Could you explain what you are trying to get at, given (a) answers already exist and (b) your code exhibits the "explicit demonstrat[ion of] bootstrap failure"? – whuber Nov 28 '20 at 23:24
  • I have updated my post. Hopefully it is clearer. I suppose I'm just looking for a "workflow" to follow in investigating failures of the bootstrap. – compbiostats Nov 28 '20 at 23:58
  • @compbiostats I don't think of simulations as answering the question of convergence. In my PhD dissertation I proved a limit theorem for the distribution of the maximum of a uniform first order autoregressive process. The actual norming constants & the exact limiting result were suggested through simulation but that didn't give an acceptable answer. A mathematical proof was required. – Michael R. Chernick Nov 29 '20 at 02:48

0 Answers0