4

I am considering this test for univariate normality which I believe, but not sure, may be just like the Q-Q test.

The procedure is like this:

  1. We compute sample mean, standard error (S.E)
  2. Then compute percent of sample values within $ \pm 1,2,3 $ S.E's from the sample mean.
  3. We then compute a chi-squared goodness of fit between expected ( using 68-95-99 % rule ) and actual.

Is this a known test? If not, how could I find out how this statistic is distributed? This is not too deep, so I don't expect it to be something new, but I am curious as to how to figure out the distribution of this statistic.

Maybe his is something really simple, in which case maybe I can delete the question. My apologies if this is so.

Karolis Koncevičius
  • 4,282
  • 7
  • 30
  • 47
gary
  • 483
  • 3
  • 12
  • 1
    This looks a lot like the chi squared test of normality (but I believe you would compute the standard deviation of your sample rather than the standard error of your mean). I think it has less power than other tests of normality though. – Michael Webb Jun 15 '18 at 21:34
  • 1
    Understanding SD is intended instead of SE, it's close to a chi-squared test, but not quite, because you are binning the data based on the data. For medium to large datasets that probably makes little difference, but for smaller ones it will. Moreover, you need a dataset in the thousands in order to populate the extreme bins with enough values for the chi-squared distribution to be a good approximation. I give the requirements needed for the chi-squared test to give correct p-values at https://stats.stackexchange.com/questions/16921/how-to-understand-degrees-of-freedom/17148?s=1|50.5927#17148. – whuber Jun 15 '18 at 22:49
  • 1
    What is the "Q-Q test," by the way? – whuber Jun 15 '18 at 22:58
  • I wonder whether it's a Shapiro-Francia (or Ryan-Joiner) test, or whether it's about seeing whether the points fall outside marked confidence bounds or whether the OP just means the informal diagnostic (i.e. examining the appearance of the plot, which isn't really a test as such). – Glen_b Jun 16 '18 at 00:23
  • @Whuber: A Q-Q plot is used to test for equality of distributions. I am not 100% , of how it works, but I think it compares the quantiles in the distribution we are testing against , in the horizontal axis vs the quantiles in the sample data in the vertical axis. – gary Jun 17 '18 at 23:53
  • A QQ plot, like most exploratory plots, doesn't "test" anything: it is used as *visual assessment.* – whuber Jun 18 '18 at 01:39
  • Ah, sorry, misread it. you probably understand this a few thousand times better than I. – gary Jun 18 '18 at 01:57

2 Answers2

3

Your description of the procedure is unclear. If you calculate the proportions in overlapping intervals those quantities will be dependent (above that caused by conditioning on the total), violating the chi-square assumption (though it should be possible to account for this issue).

If you have disjoint intervals then the statistic should be close to a chi-square as whuber notes (even though the boundaries are chosen based on the data).

I have seen something like this used before, but its power is surprisingly low (even after you take account of the fact that the chi-squared is already known to have fairly low power as a goodness of fit test) and it suffers from considerable bias against alternatives that are presumably of interest (that is, with power below the significance level) even at fairly large sample sizes.

[The chi-square has better power and less bias when the intervals are nearer to equiprobable but power is still fairly low even then]

I should mention that there's an issue with values outside $[-3,3]$, which should also be addressed.

Further note that testing goodness of fit for an assumption of normality is not particularly useful. See for example the discussion in Is normality testing essentially useless?; I particularly recommend Harvey's answer as concise and clear but covering what I see as the main issues.

Glen_b
  • 257,508
  • 32
  • 553
  • 939
  • Thanks, @Glen_b. Could you please elaborate on : "our description of the procedure is unclear. If you calculate the proportions in overlapping intervals those quantities will be dependent (above that caused by conditioning on the total), violating the chi-square assumption (though it should be possible to account for this issue)." , particularly on the last parenthetical comment? – gary Jun 18 '18 at 21:13
  • On the opening phrase: it wasn't clear whether you intended to calculate the statistic on (i) the intervals $|X|\leq 3$, $|X|\leq 2$ and $|X|\leq 1$, or (ii) the intervals $2 – Glen_b Jun 18 '18 at 22:45
  • ctd ... (though we may be able to find something even faster with a little algebra, I haven't tried that). Is there something remaining in the rest of it that I can explain? – Glen_b Jun 18 '18 at 22:51
1

There are a whole bunch of goodness-of-fit methods out there. However, I am particularly interested in this new method based on "kernel Stein measure". It is difficult to explain it here, so please see the original paper

http://proceedings.mlr.press/v48/chwialkowski16.pdf

Nick Cox
  • 48,377
  • 8
  • 110
  • 156
KiaSh
  • 71
  • 6
  • 1
    This seems more like a comment than an answer, particularly given that it doesn't address the direct issue in the question, talks about your own interests rather than the issue at hand, and then does nothing to explain what it is about. Link-only answers will become useless if the link dies. Please explain more in your answer, about what it is and how it would be helpful in relation to the question – Glen_b Jun 16 '18 at 00:10