What is the difference between effectiveness and efficacy in determining the benefit of therapy 'A' on condition 'B'?

Question

The context of this question is within a health framework i.e. looking at one or more therapies in the treatment of a condition. It appears that even well respected researchers confuse the terms efficacy and effectiveness, using the terms interchangeably.

• How can one think of efficacy versus effectiveness in a way that can help remove confusion?
• What type of study designs would be most appropriate in determining both types of results?
• Are there any authoritative journal publications, books, or web dictionaries that may help me?

Answer 1

I'm not a specialist of this domain in epidemiological studies, but it seems to me that efficacy has to do with the observed effect in a controlled setting, like a randomized controlled trial, whereas effectiveness has more to do with a larger range of outcomes or environmental factors (potentially unobserved or non manipulated in the RCT), hence it has a wider scope. At least, I've often heard of cost-effectiveness studies in pharmacoeconomics, and treatment efficacy (e.g., when comparing two treatment arms).

Quoting this article Efficacy, effectiveness, efficiency,

• efficacy is "the extent to which a drug has the ability to bring about its intended effect under ideal circumstances, such as in a randomised clinical trial"
• effectiveness is "the extent to which a drug achieves its intended effect in the usual clinical setting"

As for other references, I would suggest starting with Pitfalls of Multisite Randomized Clinical Trials of Efficacy and Effectiveness from Helena C Kraemer (Schizophrenia Bulletin 26(3), 2000), and references therein. For example, it is read that "efficacy and effectiveness are opposite extremes on a complex multidimensional continuum of decision making in research design".

Note

Coming back from the ISPOR 13th European conference, I've heard that the European Federation of Pharmaceutical Industries and Associations (EFPIA) considers there's now agreement on the following definitions:

• relative efficacy can be defined as the extent to which an intervention does more good than harm, under ideal circumstances, compared to one or more alternative interventions;
• relative effectiveness can be defined as the extent to which an intervention does more good than harm compared to one or more alternatives for achieving the desired results when provided under the usual circumstances of health care practice.

Answer 2

A standard, but not fully online, dictionary is the Dictionary of Epidemiology sponsored by the International Epidemiological Association. The latest edition is the fifth, but the fourth edition appears to be latest partly available online via Amazon. You want to look at p57-8. It says this distinction between effectiveness, efficacy and efficiency is due to Archie Cochrane in his 1972 book Effectiveness and efficiency: random reflections on health services, much of which is available on Google Books.

I won't quote too much from the above to avoid breaching copyright. I'll only note that although the Dictionary says, "ideally the determination of efficacy is based on the results of a randomized controlled trial" (my italics), you need instrumental variables estimation methods to determine efficacy if there is substantial non-compliance.

Answer 3

These terms cannot be interchanged and have a specific meaning.

Efficiency is the ability of an intervention to have effects on a specific marker in relation to a condition without it necessarily been relevant to patients. The study is done under highly controlled conditions with highly compliant participants. Clinical trials that assess efficacy are also called explanatory trials or phase I or II clinical trials.

Effectiveness is the ability of an intervention to have a meaningful effect on patients in normal clinical conditions. Clinical trial measuring effectiveness are also called pragmatic trials or phase III or IV clinical trials.

Answer 4

One place where I have seen this come up is in discussions of using "intention to treat" analysis versus an analysis that tries to get at the "efficacy" of a treatment in experiments with imperfect compliance. See the Wikipedia article on "intention to treat" (link), which includes some references.

In a run-of-the-mill randomized control trial with noncompliance, the intention to treat estimate examines only the difference between those assigned to treatment and control. However, noncompliance means that some people who were assigned to treatment may not have actually taken it, and some assigned to be in the control group may have actually received the treatment. If so, the intention to treat estimate may understate the average treatment effect that would obtain were all members of the population under study to actually take up the treatment.

When this kind of noncompliance is present, the analyst has a decision to make. She could decide to simply do the intention to treat analysis, justifying it by saying that in the real world, we cannot control compliance, and so the intention to treat analysis is more "realistic" as an estimate of what would happen were this treatment approved for use clinically. I have seen this referred to as an analysis of a treatment's "effectiveness." Or, she may use some kind of adjustment method to try to get at how people who actually took up the treatment differed from those who didn't. She could justify this by saying that what we are really interested in knowing is the biological (in the case of a medical trial) "efficacy" of the treatment, and to do so, we need to make the comparison between those who actually took the treatment and those who didn't.

The issue for an analysis of biological efficacy is, what "kind of adjustment method" is valid? The current state of the art, as I understand, is to view an experiment with noncompliance as an instrumental variables problem, a la Angrist, Imbens and Rubin (1996) (gated link), or, more generally to view the problem in terms of "principle stratification", a la Frangakis and Rubin (2002) (gated link). As such, the randomization serves as an instrument that nonparametrically identifies "efficacy" effects for at least certain subpopulations---namely, those who would comply with their treatment or control assignment. Beyond this, one could impose a more stringent model in order to identify efficacy effects, but then one may wonder, why did you bother to do a randomized experiment in the first place?

Answer 5

Efficacy deals with the results of a given health care program in a tightly defined setting, such as a radomized clinical trial (RCT), where patiens are similar in all the covariates but the treatments they are randomized to. Efficacy is linked to the notion of internal validity: usually a RCT has a high internal validity, since the study protocol excludes at the enrollment patients with particular comorbidities and includes all the procedures undergone by patients.

Effectiveness deals with normal clinical practice (as a sidelight, a definition of normal clinical practice still lacks, as it is often the case when a qualitative qualification enters a quantitative setting, such as epidemiology or biostatistics). In normal clinical practice, patients may be poorly compliant to the prescribed therapy, have tons of comorbidities, do not change their lifestyles, keep on smoking or eating junk-food; all these features are likely to reduce the efficacy on patients health state experienced in an RCT. Effectiveness links to the notion of external validity, in that it refers to patients who are visited by physicians in their everyday practice. In terms of public policies aimed at the health care sectors, effectiveness is deemed more important than efficacy, in that effectiveness mirrors what happens in the real world . This is probably why, when it comes to resourse allocation, we talk about cost-effectiveness (rarely of cost-efficacy) even though the economic analysis is pigged-back onto an RCT.

Answer 6

Efficiency is producing output at the highest level whereas effectiveness is meeting the objectives of the entreprise by using inputs productively to meet customer's needs.Then transfer in medical condotions. It would be the same !