In a RCT, what are the risk of selection bias if no allocation bias is present?

Question

In the Wikipedia page about Randomized Controlled Trials (RCT), the procedures section states that RCT minimize selection bias and allocation bias.

My understanding is that selection bias "occur if investigators can consciously or unconsciously preferentially enroll patients between treatment arms" while allocation bias "occur when covariates that affect the outcome are not equally distributed between treatment groups".

Suppose we design an RCT that has selection bias but no allocation bias.

For instance, we want to test whether a new drug for the flu is effective.

We take 100 persons, and the researcher assigns the 50 prettiest (from a physical point of view) to the treatment group (receive the drug) and the 50 ugliest to the control group (receive placebo).

The outcome is time needed to heal from the flu.

There is selection bias as the allocation is not random.

However, there is no allocation bias as physical appearance does not confound the relationship. In fact, the flu does not heal faster in prettier persons than in uglier ones (or at least, suppose that is not the case. That is, let's make no reasoning like "the prettiest ones are also the youngest and the youngest heal faster"). Let's assume that the selection variable (physical appearance) is orthogonal to the outcome.

So, what are the dangers of such a design?

Answer 1

If you merely assume away the problem, then obviously that is going to make it difficult to see the dangers of that kind of design.

Unfortunately, reality does not let us assume away statistical relationships between human characteristics. Prettier adults are in fact generally younger, healthier, thinner, more active, wealthier and more educated (the last two relative to their age cohort), etc., than uglier adults. All of those are potential covariates that would be unequally distributed between the prettier and uglier cohort, so it is a textbook case of allocation bias.

Now, perhaps the allocation bias doesn't matter. Perhaps recovery from the flu is not related to any of these things. Nevertheless, it is not something that is a safe assumption a priori, and that is precisely the danger with a selection mechanism that chooses one human characteristic as a proxy for "randomisation" --- it is generally going to be statistically related to a whole bunch of other human characteristics that may have a causal effect on the outcome.

Answer 2

Here's a simple example of how this experiment can go awry. Suppose you assign people to treatment based on beauty, which has no direct effect on recovery time. But beauty can help attract a partner that brings the patient chicken soup, thereby aiding the recovery. A second channel is that a partner will nag the patient to take the medication, which is a kind of increase in treatment intensity/dosage. This will make it hard to estimate the direct effect of the treatment since the treated group will have more partnered people relative to the control group.

Answer 3

If you STIPULATE that the allocation variable is uncorrelated with the outcome then this would be a fine (albeit odd) research design. The whole reason that randomization (say drawing from a hat) eliminates selection bias is that it is trivial to argue that the outcome of the randomization is uncorrelated with the outcome because a truly random value is not correlated with ANYTHING.

The danger is that in real life it's almost impossible to KNOW that a (non randomized) characteristic is not correlated with the outcome. But some observational research designs - such as natural experiment, regression discontinuity, or instrumental variable designs, rely on arguments that the particular characteristic that causes individuals to be assigned to the treatment or control groups is probably not correlated with the outcome.

I think a study like that might be like what you are proposing (the treatment and control groups are different - but not in ways that are correlated with the outcome), but this isn't something you would design yourself. Rather it's a method to try and achieve causal inference from observational data in situations where you did not have control over assignment.

For example, let's say students that score below a 50 on a math test are assigned to a special tutoring course, and then given a second test. To evaluate the effect of the course - even though it was not randomly assigned - a researcher might just look at those students who scored between 48.99 and 51.01 on the first test. She might argue that although test scores in general are obviously a potential confounder, whether a student scored just above or below an arbitrary cutoff is probably NOT a confounder. So if - just among this group of students who scored close to 50 on test 1 - we see higher scores on the test 2 for those students who were assigned to the tutoring course - we could argue that the result is probably not due to selection bias. This is a regression discontinuity design.