3

I am having an argument with a co-author about how to eliminate a nuisance parameter in a simple likelihood ratio test and am hoping that the community helps us settle it.

Our data $\mathbf{x}$ can be described by the likelihood functions $l(\mathbf{x};\theta_0,\eta)$ and $l(\mathbf{x};\theta_1,\eta)$ under hypotheses $H_0$ and $H_1$, where $\theta$ is the parameter of interest and $\eta$ is the random "nuisance" effect. We know the distribution function $f_\eta(\eta)$ that describes $\eta$.

The likelihood ratio is, of course, $$\Lambda(\mathbf{x},\eta)=\frac{l(\mathbf{x};\theta_0,\eta)}{l(\mathbf{x};\theta_1,\eta)}$$

Since we have $f_\eta(\eta)$, we can "average" out $\eta$ to obtain a simple test between two point hypotheses. We can then use Neyman-Pearson and calculate the value of threshold. Our argument stems from how to average it out. Since $\Lambda(\mathbf{x},\eta)$ is effectively a random variable that is a function of $\eta$, I think the following is the correct test statistic is

$$E_\eta\left[\frac{l(\mathbf{x};\theta_0,\eta)}{l(\mathbf{x};\theta_1,\eta)}\right]$$

while my co-author insists that the correct test statistic is

$$\frac{E_\eta[l(\mathbf{x};\theta_0,\eta)]}{E_\eta[l(\mathbf{x};\theta_1,\eta)]}$$ Who is right, and why?

M.B.M.
  • 1,059
  • 8
  • 18
  • 1
    When you say "we know the distribution function that describes the nuisance parameter", I'm a little confused. In the LRT, parameters are fixed but unknown. If you're describing a parameter by a distribution it sounds like you're taking a Bayesian approach instead, which would lead people to wonder, why discuss LRT at all? Perhaps more details might help to clarify the situation. – Glen_b Apr 23 '15 at 04:54
  • $\theta\in\{\theta_0,\theta_1\}$ is fixed but unknown, $\eta$ is random, and "uninteresting." Looks like van Trees has it the my co-author's way, Eq. (296) in Sec 2.5 "Composite Hypotheses"... – M.B.M. Apr 23 '15 at 05:27
  • @Glen_b: I second your point that the only cases where this makes sense are the Bayesian and random effect situations. But this does not cover all settings with nuisance parameters. – Xi'an Apr 23 '15 at 06:20
  • If $\eta\sim f_\eta(\eta)$, the "true" likelihood is $$\mathbb{E}[l(\mathbf{x};\theta,\eta)]=\int l(\mathbf{x};\theta,\eta) f_\eta(\eta)\,\text{d}\eta$$ so I would second your co-author's perspective. – Xi'an Apr 23 '15 at 06:23
  • 1
    M.B.M. Are you talking about $\eta$ being a *random effect*? I'd probably have denoted those as $\eta_i$ or something along those lines -- but in any case you should be explicit about what you mean (in your question). – Glen_b Apr 23 '15 at 08:49
  • @Glen_b I changed the question per your suggestion. Sorry for the lack of clarity. – M.B.M. Apr 23 '15 at 16:22
  • @Xi'an As I mentioned earlier, van Trees also seconds this. – M.B.M. Apr 23 '15 at 16:24

1 Answers1

3

You have to compare two likelihoods, not doing the mean of a ratio. So your co-worker is right. Since you said that you have the distribution of $\eta$ it seems to me you are using a Bayesian approach.

On the contrary, in high-energy-physics we use a different approach: we "profile" the nuisance parameters and build the profiled-likelihood ratio. In you case:

$$\Lambda(\mathbf{x})=\frac{\sup_{\eta}L(\mathbf{x}|\theta_0,\eta)}{\sup_{\eta}L(\mathbf{x}|\theta_1,\eta)}$$

This is more or less what the Tevatron experiment did to compare hypothesis (with signal / without signal), see http://arxiv.org/pdf/1001.4162v3.pdf

Ruggero Turra
  • 684
  • 7
  • 19