Weighted Least Squares Estimate

Question

Here is a problem from a practice test. Suppose that $$X_i = \mu + \epsilon_i,\quad i=1,\ldots,n\quad \epsilon_i\sim N(0,\sigma^2_1)$$ $$Y_i = \mu + \delta_i,\quad i=1,\ldots,m\quad \delta_i\sim N(0,\sigma^2_2)$$ All $\epsilon_i$'s and $\delta_i$'s are independent. The paramters $\mu, \sigma_1^2, $ and $\sigma_2^2$ are unknown. Let $\theta=m/n$, $\rho=\sigma_2^2/\sigma_1^2$. Suppose $\rho$ is known. Show that the least squares (weighted) estimator of $\mu$ is $$ \hat{\mu} = \dfrac{\rho\bar{X} + \theta\bar{Y}}{\rho+\theta}$$

MY ATTEMPT:

I can't figure out how to use the fact that $\rho$ is known. I tried $$\hat{\mu} = \text{argmin}\left\{\sum_{i=1}^n (X_i-\mu)^2 + \sum_{i=1}^m (Y_i-\mu)^2\right\}$$ and arrived that the weighted averaged $$ \hat{\mu} = \dfrac{n\bar{X} + m\bar{Y}}{n+m}$$ But again this does not use the fact that we know what the ratio $\sigma_2^2/\sigma_1^2$ is. Any ideas?

Answer 1

Consider $\mathbf{Y} = \mathbf{X}\mu + \epsilon$ where the matrices are defined as follows:

$\begin{pmatrix} X_1 \\ X_2 \\ \vdots\\ X_n \\ Y_1 \\ Y_2 \\ \vdots \\ Y_m \end{pmatrix} = \begin{pmatrix} 1 \\ 1\\ \vdots \\ 1 \end{pmatrix}\begin{pmatrix} \mu \end{pmatrix} + \begin{pmatrix} \epsilon_1 \\ \epsilon_2 \\ \vdots \\ \epsilon_n \\ \delta_1 \\ \delta_2 \\ \vdots \\ \delta_m\end{pmatrix}$

Now we consider this as a weighted least squares problem. Let $w_i = \rho$ for $i=1,\ldots,n$ and $w_i = 1$ for $i=n+1,\ldots,n+m$. Then the weighted least squares estimate of $\mu$ is

$\hat{\mu}_{WLS} = \left(\mathbf{X}^T\mathbf{W}^{-1}\mathbf{X}\right)^{-1}\mathbf{X}^T\mathbf{W}\mathbf{Y}$ where $diag(\mathbf{W}) = (w_1,\ldots,w_{n+m})=(\rho,\ldots,\rho,1,\ldots,1)$.

Solving for $\hat{\mu}$ we get

$\begin{align*} \hat{\mu} &= \dfrac{1}{\rho n + m}\left(\rho\sum_{i=1}^n X_i + \sum_{i=1}^m Y_i\right)\\ &= \dfrac{n\rho\bar{X} + m\bar{Y}}{n\rho + m}\\ &= \dfrac{\rho\bar{X} + \tfrac{m}{n}\bar{Y}}{\rho +\tfrac{m}{n}}\\ &= \dfrac{\rho\bar{X} + \theta\bar{Y}}{\rho +\theta} \end{align*}$