Simple question about the asymptotics of estimators

Question

Consider any arbitrary estimator called $\hat{M}$ (e.g., regression coefficient estimator or specific type of correlation estimator, etc) that satisfies the following asymptotic property:

$$\boxed{\sqrt{N}(\hat{M}-M) \overset{d}{\to}\mathcal{N}(0,\sigma^2)}\,\,\,\,\,\,\,\,\,\,\,\,(1)$$

which implies that our $\hat{M}$ is consistent. We also have a consistent estimator $\hat{\sigma}$, which gives rise to the asymptotic property:

$$\displaystyle \ \ \boxed{\frac{\sqrt{N}(\hat{M}-M)}{\hat{\sigma}} \overset{d}{\to}\mathcal{N}(0,1)}\,\,\,\,\,\,\,\,\,\,\,\,(2)$$

I'm wondering if I can use the $z$- or $t$-test just like normal for any such $\hat{M}$ that satisfies the above? Let $Q$ be defined as the test statistic:

$$\displaystyle \ \ \boxed{Q_\hat{M} = \frac{\hat{M}-M_{H_0}}{\sqrt{\frac{1}{N}\hat{\sigma}^2}}}\,\,\,\,\,\,\,\,\,\,\,\,(3)$$

My goal is to do the following hypothesis test:

$H_0: M = 0$

$H_a: M \not= 0$

yet the only information I have access to is $(1)$ and $(2)$, whence my question.

---

$$\underline{\text{Update}}$$

The current answers suggest that I can't always robustly $z$- or $t$-test for any such $\hat{M}$. I am reading the relevant sections of All of Statistics (Wasserman), as well as Statistical Inference (Casella & Berger). Both state that, if:

$$\displaystyle \ \ \frac{\sqrt{N}(\hat{M}-M)}{\hat{\sigma}} \overset{d}{\to} \mathcal{N}(0,1)$$

• then "an approximate test can be based on the wald statistic $Q$ and would reject $H_0$ if.f. $Q < -z_{\alpha/2}$ or $Q > z_{\alpha/2}$" (in Casella & Berger, page 492, "10.3.2 Other Large-Sample Tests")

• or, in (Wasserman, page 158, Theorem 10.13) "Let $Q = (\hat{M}-M_{H_0})/\hat{se}$ denote the observed value of the Wald statistic $Q$ $\big($where $\hat{se}$ is obviously equal to my $\sqrt{\frac{1}{N}\hat{\sigma}^2}$$\big)$. The p-value is given by:

$$p = 2\Phi(-|Q|)$$

This contradicts the existing advice since they do not state any other necessary assumptions to be able to do this legitimately (to the best of my ability to comprehend). Either;

• I have failed to understand existing answers.
• I have failed to express my original question clearly.
• I have failed to read these chapters properly.
• They are excluding thoroughness for pedagogical purposes.

I would appreciate some assistance on which option is correct. Thanks. $\big($Please go easy I am new to stats :)$\big)$.

---

Another dimension is that my intended application is $n = 3000$, so perhaps the finite sample problems are less relevant?

Answer 1

That's exactly how asymptotic results are being used in practice, e.g., in logistic regression. I would probably factor it differently as

$$\sqrt{N}\frac{\hat{M}-M}\sigma \overset{d}{\to}\mathcal{N}(0,1)$$

which shows the desired result more immediately, IMO (as mptikas mentioned in the comments, it is not kosher to have $N$ on the RHS of the asymptotic expression). The practical problem with this of course is that $\sigma$ is usually unknown, and needs to be estimated. The result, and the application, would still hold if a $\sqrt{N}$-consistent estimator is plugged in place of $\sigma$. In some applications, getting such an estimator is a non-trivial task, as is the case with say dependent data (time-series, cluster sampling, spatial data).

Update: since the asymptotic distribution is the normal rather than Student, a $z$-test is more appropriate. In practice, $t$-tests are often used instead, but coming up with the degrees of freedom is often a challenge. Besides, for most sample statistics, the finite sample asymmetry and bias are greater concerns than heavy tails, and these obviously cannot be corrected by referring the test statistic to the $t$-distribution instead of the standard normal.

Answer 2

Taking the question at its face value, the answer is no. I offer a counterexample where $\hat{M}$ approaches its estimand in distribution while its variance diverges: in such a case, the $t$ statistic must approach zero almost surely, proving it can have neither an asymptotic Normal or t distribution.

---

Consider the usual Normal setting where $\hat{M}$ is an unbiased estimator of the mean based on $N \ge 2$ iid observations of a Normal$(\mu, \sigma^2)$ variable, $(X_1, X_2, \ldots, X_N)$. Let $\beta$ be a function of $N$ to be determined later and, writing $\bar{X}$ for the sample mean, consider the estimator

$$\hat{M}(X_1,\ldots,X_N) = \beta(N)\bar{X}\ \text{ if }\ X_1\ge\max(X_1,\ldots,X_N)\ \text{ else }\ \frac{N-\beta(N)}{N-1}\bar{X}.$$

Because the first alternative in the definition of $\hat{M}$ happens with probability $1/N$ and the second with probability $(N-1)/N$, we can compute that

$$\mathbb{E}(\hat{M}) = \mathbb{E}\left(\frac{1}{N}\beta(N)\bar{X}\ + \frac{N-1}{N}\frac{N-\beta(N)}{N-1}\bar{X}\right) = \mathbb{E}(\bar{X}) = \mu,$$

showing that $\hat{M}$ is an unbiased estimator of $\mu$, and (by computing the expectation of $\hat{M}^2$ and subtracting the square of the expectation of $\hat{M}$),

$$\text{Var}(\hat{M}) = \frac{\sigma^2/N + \mu^2}{N(N-1)^2}\left((N-1)^2\beta(N)^2 + (N-1)\left(N-\beta(N)\right)^2\right) - \mu^2.$$

If we choose $\beta(N) = O(N^b)$ for $\frac{1}{2} \lt b \lt 1$, the right hand side (which is $O(N^{2b-1})$) will diverge but $\hat{M}$ will approach $\mu$ in distribution (because most of the time $\hat{M}$ will equal $ \frac{N-\beta(N)}{N-1}\bar{X}$ which is becoming arbitrarily close to $\bar{X}$).

---

In a comment, StasK has noted that this estimator $\hat{M}$ is not exchangeable in the arguments ($X_1$ plays a favored role) and asks whether that might be part of the cause of the "bad" asymptotic behavior. I do not believe so. For instance, let $s$ be the sample standard deviation and $\bar{X_{\widehat{i}}}$ be the mean of the variables with $X_i$ excluded. The distribution of $(Y_i) = (X_i - \bar{X_{\widehat{i}}})/s)$ depends only on $N$ (not on $\mu$ or $\sigma$)--it is a multivariate distribution with scaled Student t distributions as marginals--so for each $N$ we there exists a number $t_N$ for which there is a $1/N$ chance that $\max(Y_i)\ge t_N$. In the definition of $\hat{M}$, replace the condition $X_1 \ge \max(X_i)$ by $\max{Y_i}\ge t_N$. Everything works out exactly as before, but this $\hat{M}$ is invariant under permutations of the data.