Likelihood Ratio Test Equivalent with $t$ test: Difference of Two Means from Constant Variance Normal Distributions

Question

$\newcommand{\szdp}[1]{\!\left(#1\right)} \newcommand{\szdb}[1]{\!\left[#1\right]}$ Problem Statement: Suppose that independent random samples of sizes $n_1$ and $n_2$ are to be selected from normal populations with means $\mu_1$ and $\mu_2,$ respectively, and common variance $\sigma^2.$ For testing $H_0:\mu_1=\mu_2$ versus $H_a:\mu_1-\mu_2>0$ ($\sigma^2$ unknown), show that the likelihood ratio test reduces to the two-sample $t$ test presented in Section 10.8.

Note: This is Exercise 10.94 from Mathematical Statistics with Applications, 5th. Ed., by Wackerly, Mendenhall, and Scheaffer.

My Work So Far: We have the likelihood as $$L(\mu_1, \mu_2,\sigma^2)= \szdp{\frac{1}{\sqrt{2\pi}}}^{\!\!(n_1+n_2)} \szdp{\frac{1}{\sigma^2}}^{\!\!(n_1+n_2)/2} \exp\szdb{-\frac{1}{2\sigma^2}\szdp{\sum_{i=1}^{n_1}(x_i-\mu_1)^2 +\sum_{i=1}^{n_2}(y_i-\mu_2)^2}}.$$ To compute $L\big(\hat\Omega_0\big),$ we need to find the MLE for $\sigma^2:$ \begin{align*} \hat\sigma^2&=\frac{1}{n_1+n_2}\szdp{\sum_{i=1}^{n_1}(x_i-\mu_1)^2 +\sum_{i=1}^{n_2}(y_i-\mu_2)^2}. \end{align*} This is the MLE for $\sigma^2$ regardless of what $\mu_1$ and $\mu_2$ are. Thus, under $H_0,$ we have that $$\hat\sigma_0^2=\frac{1}{n_1+n_2}\szdp{\sum_{i=1}^{n_1}(x_i-\mu_0)^2 +\sum_{i=1}^{n_2}(y_i-\mu_0)^2},$$ and the unrestricted case is $$\hat\sigma^2=\frac{1}{n_1+n_2}\szdp{\sum_{i=1}^{n_1}(x_i-\overline{x})^2 +\sum_{i=1}^{n_2}(y_i-\overline{y})^2}.$$ Under $H_0,\;\mu_1=\mu_2=\mu_0,$ so that \begin{align*} L\big(\hat\Omega_0\big) &=\szdp{\frac{1}{\sqrt{2\pi}}}^{\!\!(n_1+n_2)} \szdp{\frac{1}{\hat\sigma_0^2}}^{\!\!(n_1+n_2)/2} \exp\szdb{-\frac{n_1+n_2}{2}}\\ L\big(\hat\Omega\big) &=\szdp{\frac{1}{\sqrt{2\pi}}}^{\!\!(n_1+n_2)} \szdp{\frac{1}{\hat\sigma^2}}^{\!\!(n_1+n_2)/2} \exp\szdb{-\frac{n_1+n_2}{2}}, \end{align*} and the likelihood ratio is given by \begin{align*} \lambda &=\frac{L\big(\hat\Omega_0\big)}{L\big(\hat\Omega\big)}\\ &=\szdp{\frac{\hat\sigma^2}{\hat\sigma_0^2}}^{\!\!(n_1+n_2)/2}\\ &=\szdp{\frac{\displaystyle\sum_{i=1}^{n_1}(x_i-\overline{x})^2 +\sum_{i=1}^{n_2}(y_i-\overline{y})^2} {\displaystyle\sum_{i=1}^{n_1}(x_i-\mu_0)^2 +\sum_{i=1}^{n_2}(y_i-\mu_0)^2}}^{\!\!(n_1+n_2)/2}. \end{align*} It follows that the rejection region, $\lambda\le k,$ is equivalent to \begin{align*} \frac{\displaystyle\sum_{i=1}^{n_1}(x_i-\overline{x})^2 +\sum_{i=1}^{n_2}(y_i-\overline{y})^2} {\displaystyle\sum_{i=1}^{n_1}(x_i-\mu_0)^2 +\sum_{i=1}^{n_2}(y_i-\mu_0)^2}&<k^{2/(n_1+n_2)}=k'\\ \frac{\displaystyle\sum_{i=1}^{n_1}(x_i-\overline{x})^2 +\sum_{i=1}^{n_2}(y_i-\overline{y})^2} {\displaystyle\sum_{i=1}^{n_1}(x_i-\overline{x})^2+n_1(\overline{x}-\mu_0)^2 +\sum_{i=1}^{n_2}(y_i-\overline{y})^2+n_2(\overline{y}-\mu_0)^2}&<k'\\ \frac{1}{1+\dfrac{n_1(\overline{x}-\mu_0)^2+n_2(\overline{y}-\mu_0)^2} {\displaystyle\sum_{i=1}^{n_1}(x_i-\overline{x})^2 +\sum_{i=1}^{n_2}(y_i-\overline{y})^2}}&<k'\\ \frac{n_1(\overline{x}-\mu_0)^2+n_2(\overline{y}-\mu_0)^2} {\displaystyle\sum_{i=1}^{n_1}(x_i-\overline{x})^2 +\sum_{i=1}^{n_2}(y_i-\overline{y})^2}&>\frac{1}{k'}-1=k''\\ \frac{n_1(\overline{x}-\mu_0)^2+n_2(\overline{y}-\mu_0)^2} {\displaystyle(n_1-1)S_1^2+(n_2-1)S_2^2}&>k''\\ \frac{n_1(\overline{x}-\mu_0)^2+n_2(\overline{y}-\mu_0)^2} {\displaystyle\dfrac{(n_1-1)S_1^2+(n_2-1)S_2^2} {n_1+n_2-2}}&>k''(n_1+n_2-2)\\ \frac{n_1(\overline{x}-\mu_0)^2+n_2(\overline{y}-\mu_0)^2} {S_p^2}&>k''(n_1+n_2-2). \end{align*} Here $$S_p^2=\frac{(n_1-1)S_1^2+(n_2-1)S_2^2}{n_1+n_2-2}.$$

My Question: The goal is to get this expression somehow to look like $$t=\frac{\overline{x}-\overline{y}}{S_p\sqrt{1/n_1+1/n_2}}>t_{\alpha}.$$ But I don't see how I can convert my expression, with the same sign for $\overline{x}$ and $\overline{y},$ to the desired formula with its opposite signs. What am I missing?

Answer 1

Under $H_0$, the MLEs are: $$\hat{\mu}=\frac{n_1\bar{X}+n_2\bar{Y}}{n}, \qquad \hat{\sigma}^2_0=\frac{1}{n}\left( \sum_{i=1}^{n_1}{(X_i-\hat{\mu})^2} + \sum_{i=1}^{n_2}{(Y_i-\hat{\mu})^2} \right)$$ where $n=n_1+n_2$. Under $H_a$ we get: $$\hat{\mu}_1=\bar{X},~\hat{\mu}_2=\bar{Y},\qquad\hat{\sigma}^2=\frac{1}{n}\left( \sum_{i=1}^{n_1}{(X_i-\bar{X})^2} + \sum_{i=1}^{n_2}{(Y_i-\bar{Y})^2} \right)$$ This is almost identical to your results.

Now, here is the trick (you can verify it easily): $$\sum_{i=1}^{n_1}{(X_i-\hat{\mu})^2}=\sum_{i=1}^{n_1}{(X_i-\bar{X}+\bar{X}-\hat{\mu})^2}=\sum_{i=1}^{n_1}{(X_i-\bar{X})}+n_1(\bar{X}-\hat{\mu})^2$$ The same applies for the $Y_i$'s: $$\sum_{i=1}^{n_2}{(Y_i-\hat{\mu})^2}=\sum_{i=1}^{n_2}{(Y_i-\bar{Y})}+n_2(\bar{Y}-\hat{\mu})^2$$

Using $\hat{\mu}=\frac{n_1\bar{X}+n_2\bar{Y}}{n}$, we get that

$$\bar{X}-\hat{\mu}=\bar{X}-\frac{n\bar{X}-n_1\bar{X}-n_2\bar{Y}}{n}=\frac{n_2}{n}(\bar{X}-\bar{Y})$$ and similarly $$\bar{Y}-\hat{\mu}=-\frac{n_1}{n}(\bar{X}-\bar{Y})$$

$$\hat{\sigma}_0^2=\frac{1}{n}\left( \sum_{i=1}^{n_1}{(X_i-\hat{\mu})^2} + \sum_{i=1}^{n_2}{(Y_i-\hat{\mu})^2} \right)=\frac{1}{n}\left( \sum_{i=1}^{n_1}{(X_i-\bar{X})}+ \sum_{i=1}^{n_2}{(Y_i-\bar{Y})} + n_1(\bar{X}-\hat{\mu})^2 + n_2(\bar{Y}-\hat{\mu})^2 \right)=\frac{1}{n}\left( \sum_{i=1}^{n_1}{(X_i-\bar{X})}+ \sum_{i=1}^{n_2}{(Y_i-\bar{Y})} + \frac{n_1n_2^2}{n^2}(\bar{X}-\bar{Y})^2 + \frac{n_1^2n_2}{n^2}(\bar{X}-\bar{Y})^2 \right)=\frac{1}{n}\left( \sum_{i=1}^{n_1}{(X_i-\bar{X})}+ \sum_{i=1}^{n_2}{(Y_i-\bar{Y})} + \frac{n_1n_2}{n}(\bar{X}-\bar{Y})^2 \right)$$

Now, the likelihood ratio is:

$$\lambda=\left(\frac{\hat{\sigma}^2}{\hat{\sigma}^2_0}\right)^{\frac{n}{2}}\le k,$$ which is equivalent to $$\left(\frac{\hat{\sigma}^2_0}{\hat{\sigma}^2}\right)^{\frac{-n}{2}}\le k,$$ and this is equivalent to $$\frac{\hat{\sigma}^2_0}{\hat{\sigma}^2} \le k'.$$

The last fraction is:

$$\frac{\hat{\sigma}^2_0}{\hat{\sigma}^2}=\frac{\frac{1}{n}\left( \sum_{i=1}^{n_1}{(X_i-\bar{X})}+ \sum_{i=1}^{n_2}{(Y_i-\bar{Y})} + \frac{n_1n_2}{n}(\bar{X}-\bar{Y})^2 \right)}{\frac{1}{n}\left( \sum_{i=1}^{n_1}{(X_i-\bar{X})^2} + \sum_{i=1}^{n_2}{(Y_i-\bar{Y})^2} \right)}$$ $$=1+\frac{n_1n_2}{n}\frac{(\bar{X}-\bar{Y})^2}{\sum_{i=1}^{n_1}{(X_i-\bar{X})^2} + \sum_{i=1}^{n_2}{(Y_i-\bar{Y})^2}}.$$

Substracting $1$ from both sides and dividing by $\frac{n_1n_2}{n}$, the LRT is now

$$\frac{(\bar{X}-\bar{Y})^2}{\sum_{i=1}^{n_1}{(X_i-\bar{X})^2} + \sum_{i=1}^{n_2}{(Y_i-\bar{Y})^2}} \le k''$$

This id equivalent to rejecting if

$$ \frac{\bar{X}-\bar{Y}}{\sqrt{\sum_{i=1}^{n_1}{(X_i-\bar{X})^2} + \sum_{i=1}^{n_2}{(Y_i-\bar{Y})^2}}} \le k'''$$

which has the form of a $t$-test. $\blacksquare$