0

So I finished conducting a study. Before I started data collection, I did a power analysis to make sure how many subjects I would need to collect to find an effect (using G*Power).

Now that I had a look at the data, no expected results were found (highly insignificant). My supervisor asked me to calculate the power of my results to see if we were underpowered or not. Is this possible to do? And if so, how? (e.g. for repeated and mixed ANOVAs). I myself am however skeptical if it makes sense or not (read The Abuse of Power: The Pervasive Fallacy of Power Calculations for Data Analysis by Hoenig and Heisey). Am I misinterpreting something? What are your perspectives?

Please let me know if you need more details on the study to understand my question!

Inkling
  • 265
  • 3
  • 15
  • 2
    You don't need to do any calculations to know that you are under powered. Power is the probability of finding a significant result; and you didn't find any, so the power is low. – Russ Lenth Jul 21 '17 at 01:47

2 Answers2

3

I'd put it a little differently. Once you fail to reject the null hypothesis, you're done with statistical decision making. All you know is that you have a sample with a small effect size (which you expect with probability 1-alpha if there is no real effect) OR you have committed a type II error (which you expect with probability beta if there is a real effect size).

post hoc power analysis tells you nothing about the power of your study design. You could of course try to compute the power of your original study design (unrelated to the sample results in your data). Even if that "a priori" power was "not underpowered", your statistical decision (for this particular sample result or experimental realization!) is still, at best either correct or type II error - with specified probabilities.

sss
  • 46
  • 2
0

Here is a related thread that discusses performing inference on power. You could perform this sort of analysis using historical data before conducting a prospective study. In your case you could use your observed study results as the "historical data" to perform inference on a "future study" with the same sample size as your study. What you will likely find is that the confidence interval for the treatment effect (and hence for power) is quite wide. It could be that the true treatment effect (and power) was quite high and you witnessed an unusual event (failed to reject), or your treatment effect (and power) was quite low so it's no surprise that you failed to reject.

Geoffrey Johnson
  • 2,460
  • 3
  • 12