-4

Is there a way to set chi sq test confidence from 95% to 99%?

The base function

chisq.test()

don't complain this function...

Edit:

I don't ask for p-values. My question is about confidence risk .

The alpha risk (confidence) of chisq test is by default 0.05. I would like to set it to 0.01 (99% confidence). The alpha risk is the cutoff for the statistic X. For a given Chi distribution whit 'x' degrees of freedom, the alpha risk correspond to a "X risk statistic". If 'X statistic of our data' is greater than 'X risk statistic' we rejects H0. The we can calculate p-values, that can be interpreted as the probability that our H1 is true.

I would change the confidence risk, from 0.05 to 0.01. If we change the confidence risk, the p-values changes. for example a p-value of 0.001 for 95% confidence, could be 0.2 for 99% confidence.

Andy
  • 18,070
  • 20
  • 77
  • 100
Kaervas
  • 57
  • 3
  • 1
    This question is not clear but appears to be about how to do something in R. As such it is off-topic. Please see the help center for policy on software-related questions. – Nick Cox Dec 17 '13 at 10:21
  • 2
    That function reports the exact p-value, so you can use whatever alpha you want. http://stat.ethz.ch/R-manual/R-patched/library/stats/html/chisq.test.html – D L Dahly Dec 17 '13 at 10:45
  • I am not sure what you want here, the question is very unclear. It seems like you don't want p-values. But p-values are what chisq.test reports. – Peter Flom Dec 17 '13 at 11:11
  • 1
    @NickCox I think this should stay here, at least until it is clearer what Kaervas is asking for. I think there is some statistical confusion. – Peter Flom Dec 17 '13 at 11:12
  • 2
    Your assertion about the chisq.test in R: "*The alpha risk (confidence) of chisq test is by default 0.05*" -- appears to be false. – Glen_b Dec 17 '13 at 11:25
  • 2
    Please don't crosspost: http://stackoverflow.com/questions/20629584/r-chisq-test-at-99 – Sven Hohenstein Dec 17 '13 at 12:03
  • 2
    If this question is statistical, it is about the interpretation of p-values, which is addressed at http://stats.stackexchange.com/questions/31/what-is-the-meaning-of-p-values-and-t-values-in-statistical-tests. – whuber Dec 17 '13 at 16:24
  • This question appears to be off-topic because it is about software settings. – gung - Reinstate Monica Dec 22 '13 at 03:15

1 Answers1

3

First you'll have to define "confidence risk". The only definition I could find, talks about stock's sensitivity to unexpected changes.

You're confusing a number of different topics here :

The alpha error

The alpha error or type I error is the probability of accepting the alternative hypothesis when the null hypothesis is actually true. That's a false positive. The amount of error you allow, is your own choice. Standard one uses 0.05 (i.e. you accept a probability of 5% to call something significant while it isn't). You apparently want to use 0.01

The p value

The p-value is the probability of obtaining a test statistic as extreme or more extreme than the one you observe, given that the null-hypothesis is true. In other words, it's the probability that the effect you think you see in the data, is not a real effect but the result of randomness.

This p-value is solely dependent on the assumptions you make about your test statistic. In this case, you use a $\chi^2$ test, meaning you calculate a test statistic that you assume to follow a $\chi^2$ distribution with a certain amount of degrees of freedom. Changing the p-value can only be obtained by changing the assumptions on the distribution of your test statistic, as the p-value is directly constructed from said test statistic.

Statistic critical threshold

Just as you can consider a test to be significant if the reported p-value is less than your acceptable threshold for the alpha error, you can express the exact same criteria on the scale of the test statistic. For example, a criteria of $p<0.05$ corresponds to a criteria of $\chi^2>3.841$ for a $\chi^2$ distribution with one degree of freedom. However, just as R reports a p-value but does not itself compare it to your threshold value, it also just reports a value for the $\chi^2$ statistic but does not compare it to any specific threshold value. You are free to compare it to whatever threshold value you deem appropriate.

Confidence interval

A confidence interval is based on the standard error around the estimate for which you construct that confidence interval. A 95% confidence interval means that in 95% of the experiments, the interval will contain the true value of the estimate. If you construct a 99% confidence interval, you have a wider interval because you now want the interval to contain the true value of the estimate 99% of the time. But this is yet another thing.

Joris Meys
  • 5,475
  • 2
  • 32
  • 43
  • I know *you* know what you meant when you wrote "The p-value ... [is] the probability that the variation in the data you look at is due to random variation," but as stated it's not going to be interpreted as you intended and will be considered incorrect. You ought to consider modifying or deleting that phrase. – whuber Dec 17 '13 at 17:37
  • 1
    @whuber Tried to use wording that's understood by people with a limited knowledge of statistics. Feel free to clarify, I would honestly not know how to clarify it. Continuous struggle with the students here as well... – Joris Meys Dec 17 '13 at 17:52
  • To make my concern about misunderstanding concrete, suppose a test says p=0.05. In what sense is there a 5% chance the data we have are "due to random variation"? Because randomness is part of the model, it is *certain* that random variation influenced the data values. If we hypothesize that $H_0$ is true, we can say there was a 5% chance the data *would* lie in the $\alpha=0.05$ critical region (CR). If we don't make that hypothetical, we cannot exactly determine the chance the data lie in the CR. Your characterization either makes no sense or is wrong in all three of these interpretations. – whuber Dec 17 '13 at 18:05
  • 1
    "i.e. you accept a probability of 5% to call something significant while it isn't" is not completely clear. The test will still be significant, but you reject a true H0 in this case (given that H0 is indeed true). Otherwise "significant" would be a property on the population level... I also noticed that "random variation" or "chance" is often ill understood. Maybe it could help to specify in a form such as: the deviation of the sample statistic from the expected value under H0 is due to sampling error only... – jank Dec 18 '13 at 04:44
  • Feel free to edit. I did my best, but don't know how to call "random variation" like my grandmother understands it when talking to other statisticians. Randomness is part of the model, as in the random error. But randomness is also part of the experimental setup, as random selection from the population. Hence, a non-random effect in a model can be caused by random variation nonetheless. – Joris Meys Dec 18 '13 at 08:53