Would using two one-tailed tests rather than one two-tailed test increase the significance of a hypothesis test, all else equal?

Question

Since a smaller sample is needed to reach a given level of significance for a one-tailed test than a two-tailed test, why not just run two one tailed-tests on both sides?

I understand that taking a one-tailed test, in many cases, makes unjustified assumptions about the direction of the effect under question, but why can't this simply be mitigated by taking the opposite one-tailed test as well?

It seems this way combines the best of one-tailed tests (increased significance) and two-tailed tests (avoiding bias).

You do so by doubling your overall type I error rate (i.e. you double the significance level) -- either of the one-tailed tests can reject, and they each will reject cases the other one tailed test will not. If I do the same with my two tailed test (double the significance level), I can also use a smaller sample size. — Glen_b, Feb 18 '18 at 02:47
Some related posts that may be of some use: [Why does the p-value double when using two-tailed test...](https://stats.stackexchange.com/questions/161721/why-does-the-p-value-double-when-using-two-tailed-test-compared-to-one-tailed-on), ... [Explaining two tailed tests](https://stats.stackexchange.com/questions/19195/explaining-two-tailed-tests), ... [Difference between one tailed and two tailed testing](https://stats.stackexchange.com/questions/24676/difference-between-one-tailed-and-two-tailed-testing), ... ... ctd — Glen_b, Feb 18 '18 at 02:53
... [Doubling or halving p-values...](https://stats.stackexchange.com/questions/267192/doubling-or-halving-p-values-for-one-vs-two-tailed-tests) — Glen_b, Feb 18 '18 at 02:54

score 2 · Answer 1 · edited Oct 05 '21 at 02:51

The "advantage" of one-sided tests that you refer to is that - by using the same significance level as for a two-sided test - you are essentially willing to make twice as many false rejections in your preferred direction under the null hypothesis.

It is a misconception that this is really an advantage, it is a particular choice by the researcher that favors a particular outcome. In most fields where one-sided and two-sided tests are done, it is standard to use twice the significance level for two-sided tests. At that point the "advantage" of one - sided tests goes away.

Doing two one-sided tests does indeed result in a two-sided test, but one with a type one error rate that is the sum of those of the one-sided tests.

Would using two one-tailed tests rather than one two-tailed test increase the significance of a hypothesis test, all else equal?

1 Answers1