"Pragmatic" trials: what are they?

Question

On twitter, a trialist Stuart Nicholls critiqued a recently published study by saying:

Further to the very interesting paper by Dal-Re they flag several examples that question usage of the term pragmatic. Can a phase 3, multisite, double-blind, placebo-controlled, parallel-arm, dose increment randomised trial really be called 'pragmatic'?

The research article in question is linked here. The paper by Dal-Re is here. And their recommended "pragmatism" tool is here. I might differ with Nicholls because my sense is that with the nature of the treatment and with the N of 171 patients, this is probably an underpowered study... but does that make it pragmatic (as opposed to, say, confirmatory?).

I feel like I'm missing the point. What, in fact, is a pragmatic trial? When is a trial not pragmatic in the sense that it is of no use, and when is a trial not pragmatic in the sense that it is a confirmatory trial?

Answer 1

The critical distinction between explanatory and pragmatic trials is not regarding whether a trial produces useful information. Rather, it is what use that information is for specifically: pragmatic trials are those aiming squarely at therapeutic usefulness in the clinic.

The Pragmatic-Explanatory continuum was first proposed by Schwartz and Lellouch in a 1967 paper titled "Explanatory and Pragmatic Attitudes in Therapeutic Trials" in the Journal of Clinical Epidemiology, and which was cited by the Precis-2 developers. In this article the authors present two randomized control trial scenarios testing in an anti-cancer context a drug preparatory to radiotherapy vs. radiotherapy alone. The drug is administered 30-days prior to radiotherapy in order to sensitize the patient to the effects of radiation.

1. The drug for 30 days followed by radiotherapy is tested against a 30-day wait plus radiation

2. The drug for 30 days followed by radiotherapy is tested against radiation alone beginning immediately

The first scenario, which they describe as explanatory provides "information on the effects of the key component," while the second scenario, described as pragmatic "compares two complex treatments as a whole under practical conditions".

Schwartz and Lellouch give another example distinguishing explanatory and pragmatic trials: a randomized trial where two analgesics of very similar molecular structure are compared on an "equimolecular" basis is explanatory because it is interested in the relative effect of these drugs based on the same dose; by contrast, two analgesics with radically different structures having different "optimal levels of administration" are best studied using a practical design, intended to compare the optimal effectiveness of each treatment.

The authors summarize:

The “comparison between two treatments” is a problem which is inadequately specified even in its over-all characteristics. It may imply one of at least two types of problem which are basically different.

1. The first type corresponds to an explanatory approach, aimed at understanding. It seeks to discover whether a difference exists between two treatments which are specified by strict and usually simple definitions. Their effects are assessed by biologically meaningful criteria, and they are applied to a class of patients which is rather arbitrarily defined, but which is as likely as possible to reveal any difference that may exist. Statistical procedures used in determining the number of subjects and in assessing the results are aimed at reducing the probabilities of errors of the first and second kind.

2. The second type corresponds to a pragmatic approach, aimed at decision. It seeks to answer the question-which of the two treatments should we prefer? The definition of the treatments is flexible and usually complex; it takes account of auxiliary treatments and of the possibility of withdrawals. The criteria by which the effects are assessed take into account the interests of the patients and the costs in the widest sense. The class of patients is predetermined as that to which the results of the trial are to be extrapolated. The statistical procedures are aimed at reducing the probability of errors of the third kind (that of preferring the inferior treatment); the probability of errors of the first kind is 1.0.

Schwartz, D. and Lelluch, J. (1967). Explanatory and pragmatic attitudes in therapeutic trials. Journal of Clinical Epidemiology, 20:637–648.

Answer 2

The Schwartz & Lellouch paper mentioned by Alexis, originally published (1967) in J Chron Dis, was reprinted in 2009 in a J Clin Epi issue that took up this theme in a number of papers [1–8].

Of these papers, I found Karanicolas et al [5] particularly helpful for introducing a new distinction that illuminates (and helps to restore) the original sense of Schwartz & Lellouch. (See also the ensuing exchange [6–8] with Oxman et al.) In brief, [5] argues that Schwartz & Lellouch's original focus on constrasting the purposes of trials has been lost in subsequent usage. To restore that focus, [5] articulates a more refined mechanistic-practical contrast, advancing 'practical' trials as those useful for individual-level decision making (doctor-patient) as against 'pragmatic' trials that may appeal to policy-makers wishing to influence the clinical encounter from behind their desks at insurance companies or government agencies.

The intrinsically political aspects of this matter have, no doubt, contributed to muddying the concepts. There is an ongoing tension within medicine, between efforts to centrally plan and control the doctor-patient encounter and efforts to preserve (and increasingly, to restore) the traditional character and independence of the doctor-patient relationship. Probably the phenomenon of pragmatic trials cannot be fully understood without appreciating arguments against industrialized medicine such as Victor Montori (a coauthor of [5]) now prominently advances in his book, Why We Revolt: A patient revolution for careful and kind care.

1. Schwartz D, Lellouch J. Explanatory and pragmatic attitudes in therapeutical trials. J Clin Epidemiol. 2009;62(5):499-505. doi:10.1016/j.jclinepi.2009.01.012.
2. Zwarenstein M, Treweek S. What kind of randomized trials do we need? J Clin Epidemiol. 2009;62(5):461-463. doi:10.1016/j.jclinepi.2009.01.011.
3. Thorpe KE, Zwarenstein M, Oxman AD, et al. A pragmatic–explanatory continuum indicator summary (PRECIS): a tool to help trial designers. J Clin Epidemiol. 2009;62(5):464-475. doi:10.1016/j.jclinepi.2008.12.011.
4. Maclure M. Explaining pragmatic trials to pragmatic policymakers. J Clin Epidemiol. 2009;62(5):476-478. doi:10.1016/j.jclinepi.2008.06.021.
5. Karanicolas PJ, Montori VM, Devereaux PJ, Schünemann H, Guyatt GH. A new "Mechanistic-Practical" Framework for designing and interpreting randomized trials. J Clin Epidemiol. 2009;62(5):479-484. doi:10.1016/j.jclinepi.2008.02.009.
6. Oxman AD, Lombard C, Treweek S, Gagnier JJ, Maclure M, Zwarenstein M. Why we will remain pragmatists: four problems with the impractical mechanistic framework and a better solution. J Clin Epidemiol. 2009;62(5):485-488. doi:10.1016/j.jclinepi.2008.08.015.
7. Karanicolas PJ, Montori VM, Devereaux PJ, Schünemann H, Guyatt GH. The practicalists’ response. J Clin Epidemiol. 2009;62(5):489-494. doi:10.1016/j.jclinepi.2008.08.013.
8. Oxman AD, Lombard C, Treweek S, Gagnier JJ, Maclure M, Zwarenstein M. A pragmatic resolution. J Clin Epidemiol. 2009;62(5):495-498. doi:10.1016/j.jclinepi.2008.08.014.

Answer 3

An efficacy trial is more likely to determine the relative benefit of A vs B, but only in a setting so artificially constructed that its applicability to the real world is questionable. For example, patients in an efficacy trial may have repeated clinic visits and adverse event capture tools not present in the real world. However, because of the visits we can be assured that A and B were given throughout the trial and outcomes are more likely to be accurately measured. An efficacy trial attempts to determine the true, cosmic difference between A and B.

A pragmatic or effectiveness trial obtains external validity by treating a broad group of patients with realistic regimens, but may suffer in its ability to isolate the A/B difference due to the inherent heterogeneity of the real world. For example, an effectiveness trial may compare A to B in patients using usual clinical follow-up as recorded in unstructured visits or using administrative data to gather outcome status. Because patients had the follow-up they would in the real world, we may believe the treatment regimen is more repeatable outside the research setting. However, important events may be missed due to the lack of structure. An effectiveness trial asks whether providers should write a prescription for A or B here on Earth.