arbitrariness in bootstrap bias estimation

Question

The bootstrap estimates bias by applying the "plug-in" principle to $$E(\hat{\theta}_n) - \theta$$ I got this knowledge from p.124 of Efron, Tibshirani, 1994.
equation(10.1) $\text{bias}_F=E_F[s(\mathbf{x})] -t(F)$ and equation(10.2) $\text{bias}_\hat{F} = E_\hat{F}[s(\mathbf{x}^*)] - t(\hat{F})$

the "plug-in" of the first term has definitive meaning, since $\hat\theta_n$ is a prescribed statistics on the sample. We simply need to take the expectation of it with respect to the empirical distribution.

the "plug-in" of the second term however is rather confusing. Since there are an infinite number of ways to write a given distribution parameter as a functional of the distribution. For example, take $\theta$ to be the decay rate parameter of the exponential distribution, thus $f(X) = \theta e^{-\theta X}$, and one could get $\theta = 1/E(X)$ as well as $\theta = 1/\sqrt{D^2(X)}$, these are different functionals of $f(X)$ and would lead to different plug-in estimate of $\theta$ on finite sample.

This might be a naive question but I hope I've clarified my confusion.

---

Since I didn't get an answer after half a year, I though maybe I mis-stated the question, So I'll restate it: What's the definition of a "parameter" as repeatedly used in Efron's book? Is it a functional by definition? Or is there a standardized way to write every "parameter" as a functional? Can you give some more examples of a "parameter" (other than the "mean" and "variance")?

Quoted from page 124 of "A introduction to the bootstrap":

... . We want to estimate a real-valued parameter $\theta = t(F)$. For now we will take the estimator to be any statistic $\hat{\theta}=s(x)$

Answer 1

This is best explained using the figure from the Second Thoughts on the Bootstrap paper by Bradley Efron.

In real world, there is a distribution $P$ (denoted as $F$ in the book), where the data $\mathbf{x}$ comes from the distribution. We also have the estimator $t$, using it we can obtain population parameter $\theta = t(P)$. We also can use statistic $s$ to estimate $\theta$ from the sample obtaining $\hat{\theta} = s(\mathbf{x})$.

In bootstrap world, we obtain a sample $\mathbf{x}^*$ from bootstrap distribution $\hat{P}$, we can as well obtain the population estimate of the bootstrap distribution $t(\hat{P})$, or sample statistic from the bootstrap sample $s(\mathbf{x}^*)$.

The plug-in principle means that you substitute the distribution $P$ with bootstrap distribution $\hat{P}$, and sample $\mathbf{x}$ with bootstrap sample $\mathbf{x}^*$. We are allowed to do this because bootstrap imitates sampling $\mathbf{x}$ from $P$, by applying equivalent sampling procedure to sample $\mathbf{x}^*$ from $\hat{P}$.

What follows, if we have the definition of bias

$$ \text{bias}_P(\hat{\theta}, \theta) = \text{bias}_P = E_P[s(\mathbf{x})] - t(P) $$

then we substitute

$$ \text{bias}_\hat{P} = E_\hat{P}[s(\mathbf{x}^*)] - t(\hat{P}) $$

To give an example, say that you want to assess the bias of sample mean $s(\mathbf{x})$ as an estimator of population mean $t(P)$, then you use bootstrap to sample from the distribution $\hat{P}$ (i.e. sample with replacement from $\mathbf{x}$), calculate the sample means on the bootstrap samples $s(\mathbf{x}^*)$, and compare their expected value (mean), with the population mean of the bootstrap distribution $t(\hat{P})$. The "population mean" from bootstrap distribution is equivalent to sample mean of $\mathbf{x}$, since distribution $\hat{P}$ is created by sampling from $\mathbf{x}$, so $\hat{\theta} = s(\mathbf{x}) = t(\hat{P})$.