If the level of a test is decreased, would the power of the test be expected to increase?

Question

If we have some test, and we decrease its level, would the power be expected to increase? I have given some thought to this question before but haven't been able to convince myself of the correct answer as the calculation of power seems to be multi-step and so I cannot make a direct judgement call as I would if we had something like an inverse function. Any help would be greatly appreciated!

Answer 1

The likelihood of making a type 1 error v. a type 2 error is inversely proportional. Thus, if you make your rejection of the null less stringent, all else being equal, the power of your test should increase.

From Wikipedia on Statistical Power: “One easy way to increase the power of a test is to carry out a less conservative test by using a larger significance criterion, for example 0.10 instead of 0.05. This increases the chance of rejecting the null hypothesis (i.e. obtaining a statistically significant result) when the null hypothesis is false, that is, reduces the risk of a Type II error (false negative regarding whether an effect exists).”

Answer 2

The level is of a test is its significance level, $\alpha$. Decreasing the level means making $\alpha$ smaller.

If you make $\alpha$ smaller, you make $\beta$ larger ... and power is $1-\beta$.

It's easiest to see what's going on in the context of a power curve, for example, with a one-sample t-test. If you draw the power against the difference between the true population mean and the hypothesized population mean, $\delta=\mu-\mu_0$, you get a curve that is at $\alpha$ when $\delta=0$ and increases as $\delta$ gets further away from 0. If you reduce $\alpha$, you "pull the curve down":

As we move from blue (10% significance level) to dark red (5%) to green (1%), ceteris paribus, the whole power curve moves down.