For a seminar, I am preparing an overview of advantages and shortcomings of randomized control trials (RCT; one shortcoming is, for example, often a limited external validity).
To illustrate potential problems with RCTs, I would like to give some examples of what can go wrong during the implementation of a randomized control design. I remember once being told about the following story of a flawed randomized controlled trial. It goes roughly as follows (I do not remember it exactly):
In the early days of the development of HIV drugs, an AIDS diagnosis often meant an agonizing death. Patients were accordingly in despair. At one point there was a promising drug (which later was shown to have indeed some beneficial effects). This drug was tested for efficacy against a placebo in a double blinded randomized trial. There were no obvious flaws in the randomization protocol or how it was implemented. The results of the initial trials were however a rather precise zero effect. How this could be the case?
The answer is the following (or similar to that):
Patients had to show up repeatedly to receive the treatment or placebo. Patients obviously did not know whether they are in the treatment or the control group. They just knew what the treatment pill could perhaps save or prolong their life. So they took their chance: After receiving the pills, all participants met. In order to ensure that they receive the treatment pill (on average) at least half of the time, they throw all pills into a basket and mixed them again.
I hope I remember this example correctly. I think this is an interesting story about what can go wrong with randomization protocols. However, I cannot find any reference confirming this story!
Could anyone provide me with a source (ideally including further information)?
Are there other similar examples?
