2

How can a test statistic be found?

Suppose that we have a sample, and we want to have a simple hypothesis. We know that under $H_{0}$ one of the summaries (which is a random variable) of this sample has a known distribution. Is it correct to use this summary as a test statistic?

Glen_b
  • 257,508
  • 32
  • 553
  • 939
Toney Shields
  • 742
  • 4
  • 20
  • 1
    Could you try to make your question more precise? Providing example illustrating your question would probably help. – Tim Mar 22 '16 at 09:37
  • The short answer is yes (but I definitely agree with @Tim that more detail is needed). For your test to have nontrivial power, the summary would also need to have another distribution under the alternative hypothesis. – Christoph Hanck Mar 22 '16 at 09:45
  • I don't have an example that comes to my mind right now, but let's say we have a sample of population, and that the mean is a summary for this population, let's also assume, for the sake of explanation, that the mean is a random variable $S$ (which is not). Under $H_{0}$ the distribution of $S$ is $A$. Can I use $S$ directly as a test statistic? I mean by that computing the observed value of $S$ and then comparing it to the quantile of the distribution of $A$ in order to determine if I should accept or reject $H_{0}$. – Toney Shields Mar 22 '16 at 09:59
  • @ToneyShields basically "yes", if you know the null distribution, you can compare some value to it's null to run a test. But it is not totally clear from your question what do you mean -- an by this you could get an answer that answers different question that you're asking. – Tim Mar 22 '16 at 10:44
  • Okay, can you please reffer me to some articles that propose their own statistic tests, I want to see what do they do in order to come up with the test statistic, how do they know it's distribution. – Toney Shields Mar 22 '16 at 10:59

1 Answers1

2

We know that under H0 one of the summaries (which is a random variable) of this sample has a known distribution. Is it correct to use this summary as a test statistic?

It kind of depends partly on what you mean by "correct".

You can use a statistic whose distribution is known under the null to perform a test of known size; in that sense it will be "correct".

However, unless that summary statistic relates to the hypothesis you're testing, it may make for a useless test (that is, a test can be 'correct' in the sense of having the desired significance level, but be of no practical use).

So for example, under an assumption of normality and given a null hypothesis about the mean, I know the distribution of the sample variance, but it's useless to use it for my test about the mean. You need a statistic that will have a different distribution under the alternative, and specifically, have a rejection region that the test statistic will fall into more often under the alternative than under the null.

Even when the statistic does "relate" to the population quantity (or quantities) in the null, you would want to consider the properties of the resulting test (power, test bias and so on) to assess its suitability, especially where alternative possibilities for test statistics can be identified.

[Note that you don't necessarily need to know the distribution of your test statistic under the null to use it as a test statistic; for example permutation tests or other resampling tests (e.g. bootstrap tests) can often be performed.]

Glen_b
  • 257,508
  • 32
  • 553
  • 939
  • Thanks, I understand better now. I have another question if you don't mind: Suppose that we have a test statistic and we know it's distribution, do we need to know the asymptotic distribution (I mean doesn't all the distributions tend to have a normal one when the size of the sample tends to infinity?) – Toney Shields Mar 23 '16 at 09:03
  • You don't need an asymptotic distribution unless for some reason you can't use the exact one for some large sample size you have. Not all distributions tend to the normal, asymptotically. – Glen_b Mar 23 '16 at 09:07
  • Thank you very much, you've been very helpful I apreciate it. One last thing, can you please reffer me to some articles that propose their own statistic tests? I want to see what do they proceed in order to come up with the test statistic and what do they doin order to know it's distribution. – Toney Shields Mar 23 '16 at 09:30
  • Well okay, but from the sound of your post they'll mostly be beyond you. 1. [Here](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4138888/)'s a paper I was looking at recently which derives a way to approximate quantiles of the distribution of the intraclass correlation coefficient. (In the paper they use it for confidence intervals rather than hypothesis tests, but it could be used for a hypothesis test.) $\:$ ... ctd – Glen_b Mar 23 '16 at 10:34
  • ctd... 2. Something simpler -- [here](http://l.academicdirect.org/Horticulture/GAs/Refs/Pearson_1900.pdf)'s Pearson's 1900 paper in which he derives the chi-square distribution for a chi-square goodness of fit test (though he does already take the multivariate normal approximation to the multinomial as a given. However, you may want to read Plackett's [paper](http://mathfaculty.fullerton.edu/sbehseta/Pearson-chisq-placket.pdf) – Glen_b Mar 23 '16 at 10:34
  • 3. Not a paper, but [here](http://stats.stackexchange.com/questions/81151/likelihood-ratio-for-two-sample-exponential-distribution) there's a questions about deriving a likelihood ratio test for equality of scales in exponential distributions. There's some detail on the distribution of the ratio of the sample means [here](http://stats.stackexchange.com/questions/76689/how-to-compare-the-mean-of-two-samples-whose-data-fits-exponential-distributions/76695#76695), which comes up in the previous question. – Glen_b Mar 23 '16 at 11:00
  • If you need something more specific, perhaps a new question is in order. – Glen_b Mar 23 '16 at 11:00
  • once again, thank you very much, I really apreciate it. You deserve more reputation, can't I add reputation to your comments? – Toney Shields Mar 23 '16 at 14:31
  • You can't add reputation to comments (you can upvote them, but comments are ephemeral and can be expected to disappear at any time). – Glen_b Mar 23 '16 at 22:18