0

In randomized controlled trials, mortality is often compared between treatment and placebo using the Kaplan-Meier estimator (for instance, the remdesivir trial ACTT-1). The trial design could have a 1:1 ratio of treatment vs placebo, or some other ratio.

This question assumes the number of trial participants, $N$, is fixed.

The question is what ratio of placebo:treatment, if any, minimizes the expected p-value under the assumption the treatment has a mortality effect? Since p-values, especially p-value thresholds for statistical significance, play a traditional role in rejecting the null hypothesis (for instance no mortality benefit until p<0.05), is it possible to design a trial for fixed $N$ explicitly to minimize expected p-value with some ratio of placebo:treatment? Do some ratios of placebo:treatment, such as 2:1, 1:1, or 1:2, have different expected p-values under the assumption of an effect?

OrangeSherbet
  • 167
  • 8

1 Answers1

2

1:1 is best.

You say "mortality is often compared between treatment and placebo using the Kaplan-Meier estimator" but in the ACTT-1 trial the comparison was a stratified logrank test. Either way, you will end up with a test statistic that has a $N(0,1)$ distribution under the null and a $N(\delta,1)$ distribution under the alternative. This could be based on differences in the Kaplan-Meier estimator at specified time, or on the logrank test, or on area under the Kaplan-Meier curves (restricted mean) or something else.

The treatment:control ratio affects the test only through its effect on $\delta$. A higher value for $\delta$ means a lower expected $p$-value, and also means higher power at any $p$-value threshold. The ratio that is optimal for one purpose is optimal for them all.

And we know 1:1 is optimal for power (eg here).

That's why you rarely see trials with different ratios: some exceptions are multi-arm trials (where the 'control' arm may be larger, as here), or adaptive trials where the ratio changes over time as more is learned.

Thomas Lumley
  • 21,784
  • 1
  • 22
  • 73
  • I think I understand now from the 1:1 optimal power example shared, but not sure about $N(0,1)$ and $N(\delta,1)$ test statistic distribution: what is the test statistic, why is it dimensionless, why variance equal to 1? From what I understood from the shared example, the mean mortality rate in the placebo group $\mu_1$ and the treatment group $\mu_2$, whatever the test is, have respective distributions, and thus the difference in means $\mu_1-\mu_2$ between the groups has its own distribution, one whose variance/width and p-value for any given effect size is minimized with equal allocation. – OrangeSherbet Jun 30 '20 at 05:06
  • 1
    Yes, except that they didn't compare mean mortality rates. The test might start off as as a difference in proportions, say. It would then be divided by an estimate of its standard deviation to get a dimensionless statistic that has a standard Normal distribution under the null, and (to a good approximation) a Normal with some other mean and the same variance under the alternative. – Thomas Lumley Jun 30 '20 at 05:15