What does it mean to determine the power of your experiment via simulation? Does it mean to run the experiment over and over again and count how many times a specific set of null hypotheses are rejected given that they are false?
Asked
Active
Viewed 156 times
2
-
2You might want to take a look at this: [simulation-of-logistic-regression-power-analysis-designed-experiments](http://stats.stackexchange.com/questions/35940/), & this: [calculating-statistical-power](http://stats.stackexchange.com/questions/21237/). – gung - Reinstate Monica Apr 10 '13 at 14:21
-
1Not to run an actual experiment repeatedly, but a 'simulation experiment' - to use simulation of random variables to emulate the characteristics of the assumed testing situation under whatever effect size(s) you're interested in - to have computer-generated ersatz experiments that tell you about the proportion of rejections under a given set of assumptions. But yes, you count the proportion of rejections at a particular effect size to compute the power at that effect size. – Glen_b Apr 10 '13 at 14:32
-
Just out of curiosity, for a split plot experiment, there are two experimental units, whole plot units, and split plot units. Are there multiple power terms for the subplot and whole plot effects? – phil12 Apr 10 '13 at 14:47
-
1Be aware that a "false null hypothesis" is not a definite state of affairs and so requires further specification. For instance, when comparing two proportions $p$ and $q$, the null hypothesis $p=q$ leads to a definite distribution of a test statistic, whereas its negation $p\ne q$ leaves us unable to do anything quantitative until we specify exactly *how much* $p$ differs from $q$. Thus *power depends on "effect size".* Among other things, this means that computing power via simulation typically requires many separate simulations across a range of effect sizes. – whuber Apr 10 '13 at 16:01