6

Say I'm testing $H_0: Y \sim \text{Exp}(1)$ against $H_1: Y \sim \text{U}(0, 1)$. I believe this gives me the following likelihood ratio test:

$$ t^*(y) = \frac{p_1(y)}{p_0(y)} = \frac{1}{e ^ {-y}} = e ^ {y} $$

The problem is defining the support for this statistic. Since $p_0(y)$ is defined $\forall y > 0$ and $p_1(y)$ $\forall y \in (0, 1)$, I don't know what to make of the support of the function $t^*(y)$, could I simply make it the intersection between $y > 0$ and $0 < y < 1$ (i.e., $0 < y < 1$)?

Waldir Leoncio
  • 2,137
  • 6
  • 28
  • 42

2 Answers2

4

This statistic weighs evidence for the two hypotheses by comparing their probability densities at the observed value of $y$. Because the denominator could be zero in this situation, we have to consider two possibilities:

  1. The denominator is positive. This means that $H_0$ assigns a positive chance to any tiny neighborhood of $y$. It occurs when $0 \lt y$. There is no problem with a division by zero. In terms of the indicator function $\mathcal{I}$, a formula for the likelihood ratio is $$\frac{\mathcal{I}_{(0,1)}(y)}{e^{-y}}.$$ This equals $e^y$ for $0 \lt y \lt 1$ and otherwise is zero.

  2. The denominator is zero. This means $H_0$ assigns no probability density to $y$. There are two possibilities:

    • $H_1$ assigns no probability density to $y$, either. Thus, this $y$ has no chance of being observed under either hypothesis. We needn't consider this any further. The set of $y$ for which this is the case is the intersection of the complements of the supports of the hypotheses: the non-positive real numbers.

    • $H_1$ assigns some probability density to $y$. Thus, this $y$ is possible under $H_1$ but not under $H_0$. The conclusion is obvious. As a convention we may use values in an extended Real number line $\{-\infty, \infty\}\cup \mathbb{R}$ to designate such likelihood ratios (or their logarithms); here we would say that the likelihood ratio (and its log) is $\infty$.

To summarize, let $S_i\subset\mathbb{R}$ be the supports of the hypotheses. Then the likelihood ratio must be considered a function whose domain is the union of supports $S_0\cup S_1$ which takes values in the extended positive reals $[0,\infty)\cup\{\infty\}$. The log likelihood ratio takes values in the extended reals $\mathbb{R}\cup\{-\infty,\infty\}$, with $\log(0)$ defined to be $-\infty$.

When $S_0=S_1$, the zero-denominator case has no chance of happening, regardless of the hypothesis, and we may dispense with using the extended reals if we wish. This is a frequent assumption in likelihood ratio settings.

whuber
  • 281,159
  • 54
  • 637
  • 1,101
  • When trying to choose between two models we end up with a 0 denominator, what is our conclusion? Since all models are wrong and some models are useful, if there is a model that assigns a 0 density for some data points but in general extremely accurate for many other situations we should totally disregard this model? – Cagdas Ozgenc Jun 14 '16 at 14:38
  • @Cagdas That's a good question. What you are positing is a situation in which our observation cannot be explained by any of the models under consideration. What action to take depends on many things, but could include scrutinizing the data (maybe they are incorrect) or augmenting the model. Both have interesting implications in formal testing situations, because it could be argued they ought to affect p-values or posterior distributions. Let us hope, then, that such circumstances arise during exploratory data analysis or model-building stages of a study! – whuber Jun 14 '16 at 14:45
2

Actually both densities are defined over the whole line, they're just 0 elsewhere than the places you mention.

You have to think carefully about the density across at least the +ve half-line -- your ansẃer defines what you get when $0<y<1$, but what's the LR when y=4.3?

That could happen, if the distribution really were exponential, so you have to consider it.

Glen_b
  • 257,508
  • 32
  • 553
  • 939
  • 2
    What happens when denominator is 0? – Cagdas Ozgenc Jun 14 '16 at 11:54
  • @CagdasOzgenc In this particular case the LR would be 0/0, but the fact that this is undefined is of no great concern since (according to both our hypotheses) this won't ever occur. If it did, we would have to regard both hyotheses as wrong. If the denominator were 0 but the numerator were not, we'd have to extend the codomain of the LR function to include $\infty$, but this would cause no additional difficulty – Glen_b Jun 14 '16 at 14:39
  • Sorry, I'm not sure what you're asking. Are you asking about where the numerator could be non-zero but the denominator is zero? – Glen_b Jun 14 '16 at 14:45
  • It doesn't matter. As I can take the inverse. Let's assume there are zeros in the denominator and nominator but they don't overlap. What will the conclusion regarding which model to choose? – Cagdas Ozgenc Jun 14 '16 at 14:46
  • @Cagdas You'll be evaluating the function (the LR) at a particular point. Either numerator and denominator are finite, or one is finite and the other is zero (there's also the 0/0 case but you're ruling that one out for now). The first case is the usual case. The case with numerator zero is covered in my answer. The case with denominator 0 is covered in my comments above (but doesn't apply to this question). Or see whuber's answer, which supplies more detail – Glen_b Jun 14 '16 at 14:51
  • I am not asking about the situation in the original question. In general you will not evaluate this for a single data point. You will apply the entire data set. Some of them will have 0 in the nominator some will have a 0 in the denominator. And if we have independence assumption the entire denominator will collapse to 0 due to multiplication of probabilities. So which model is better? – Cagdas Ozgenc Jun 14 '16 at 14:55
  • 1
    @Cagdas The likelihood is a function of the entire data set, but the likelihood in the numerator and in the denominator is *one single value*. The LR is a single value divided by a single value; for a given data set the LR is a single value. (In the particular situation being asked about in this question, however, it looks like n=1) – Glen_b Jun 14 '16 at 14:58
  • Let us [continue this discussion in chat](http://chat.stackexchange.com/rooms/41156/discussion-between-cagdas-ozgenc-and-glen-b). – Cagdas Ozgenc Jun 14 '16 at 15:00