A way to show Simpson's paradox is happening in the data

Question

I'm not entirely sure what I am talking about is Simpson's paradox, because an opposite relationship does not appear when you combine two data sets, but merely a different one. Still, I think it is some variation of the paradox.

I present below some data I am working with, made generic as X and Y.

Here are the relationships in question uncombined:

When the two data sets are combined, a strong negative relationship appears between. Clearly, uncombined, there is not a strong negative relationship in both data sets. For my study, it's important that there is not the same strong negative relationship combined as there is uncombined.

What I want to know is there any kosher way to say—or to show statistically—in a publishable paper that a strong negative relationship may only appear when these datasets are combined due to the following: The datapoints for Y in the green set are generally higher than the Y datapoints in the yellow set, and the datapoints for X are generally smaller for green than they are for yellow?

I have asked my statistician working with me on this, and he does not know of any such way, but he advised me to seek out a way to do it, if feasible.

Here are the data for these two sets, color coded:

Y       X            Color
29.2    3.822954823  orange
45.4    4.446472019  orange
37.8    4.364963504  orange
18.6    4.154740061  orange
36.2    3.449355433  orange
22.2    4.426129426  orange
49.8    3.765931373  orange
28.6    4.552311436  orange
54.4    4.270718232  orange
49.4    4.668501529  orange
18      4.32480195   orange
41.6    3.733698964  orange
59.6    3.371865443  green
52.3    3.404674047  green
76.8    3.20353443   green
41.8    3.198529412  green
64.2    3.352293578  green
34.4    3.559021407  green
69.3    4.107033639  green
62.4    3.363302752  green
45      3.109489051  green
59.2    3.8          green
38.1    3.178023327  green
15.8    4.550671551  green
31.4    3.823887873  green
43.1    3.5          green
72.1    3.613040829  green
61.3    3.386029412  green
59.9    3.549664839  green

Answer 1

You can call it "confounding" or "omitted variable bias":

1. Confounding: The statistical relationship between $X$ and $Y$ is confounded by the grouping factor "color". [Maybe you remember that a confounder is a variable which is both related to $X$ and $Y$, which is the case in your picture.]

2. Omitted variable bias: The estimated statistical relationship between $X$ and $Y$ suffers from an omitted variable bias if the grouping factor "color" is omitted.

Both are describing the same problem in different words. Simpson's paradoxon can be viewed as an extreme case of those phenomena.

When I reread your post I was somehow unconfortable about my answer because usually, we want to avoid confounding/omitted variable bias but it seems that you are preferring the "crude", unadjusted relationship between $X$ and $Y$.

Usually, we first describe the crude relationship and then revise it by confounder adjusting it. But I am sure also the converse can be useful in some cases.

Now, how can you "quantify" confounding? In a regression setting, you would compare the estimated slopes between two regressions both run on the pooled data set: One by regressing $Y$ on $X$ and a more complex one by regressing $Y$ on $X$, $\text{Color}$ and eventually on the interaction of $X$ $\text{Color}$ (then you have two comparisons to make). It is a very broad topic, so it is difficult to provide a complete answer.

Answer 2

It isn't literally Simpson's paradox, because the definition of that is that the sign of the relationship between X and Y changes upon adding another variable. Since the sign doesn't change, it doesn't quite meet the definition and so, it isn't a case of Simpson's paradox.

However, Simpson's paradox is just a special case of a more general phenomenon. If the third variable, we could call it Z (or in our case color), is not perfectly uncorrelated with X and Y, then the apparent relationship between them will change when we take Z into account. Whether or not the sign flips is dramatic, but not necessarily substantively important. Consider a case where the correlation changes from $r = .1 \rightarrow r = -.1$, and compare that to a case where the correlation changes from $r = .9 \rightarrow r = .1$. In which case might the change have been more important? I suppose it depends on the subject matter, but my first guess would be the latter. For a fuller treatment of this topic, it may help to read my answer here: Is there a difference between 'controlling for' and 'ignoring' other variables in multiple regression?

At any rate, you want to show that this is what is going on in your data. To establish this, you just have to show that the correlations between color and X, and between color and Y, are not exactly 0 in your sample (you don't need hypothesis tests). You can do that by reporting the two correlations. If you want to make a statement about the values of X and Y differing by color in the population, you could do a couple of t-tests. If you wanted to determine if the relationships between X and Y were the same in both color groups, you would fit a multiple regression model with an interaction and see if the slopes and intercepts were close enough for your purposes to state that the differences 'might as well be 0 for all you care'. (Note that this is an equivalence test, which is an inversion of the more typical logic of hypothesis testing. To understand this more fully, it may help you to read my answer here: Why do statisticians say a non-significant result means “you can't reject the null” as opposed to accepting the null hypothesis?)

Here are examples of the analyses I've discussed, coded in R:

## reading in your data:
d = read.table(text="Y       X            Color
               29.2    3.822954823  orange
               ...
               59.9    3.549664839  green", header=T)

## showing the correlations aren't 0: 
with(d, cor(Y, Color=="green"))  #  0.4840144
with(d, cor(X, Color=="green"))  # -0.65251

## testing if the groups differ in the population:
t.test(Y~Color, data=d, alternative="greater")
#  Welch Two Sample t-test
# t = 3.009, df = 26.74, p-value = 0.002828
t.test(X~Color, data=d, alternative="less")
#  Welch Two Sample t-test
# t = -4.4478, df = 23.333, p-value = 8.952e-05

## assessing whether the relationship b/t X & Y are similar in the 2 colors:
m = lm(Y~X*Color, d)
summary(m)
# ...
# Coefficients:
#               Estimate Std. Error t value Pr(>|t|)   
# (Intercept)     105.41      35.84   2.941  0.00695 **
# X               -15.07      10.09  -1.493  0.14785   
# Colororange     -61.13      60.89  -1.004  0.32499   
# X:Colororange    13.07      15.51   0.843  0.40739   
# ...

## the linear relationships between X & Y for the two colors are: 
# green:  105.41 - 15.07*X  
# orange:  44.28 -  2.00*X  
## the differences in the intercepts & slopes, w/ 95% CIs are:
difs           = data.frame(coef(m)[3:4], confint(m)[3:4,])
colnames(difs) = c("difference", "2.5 %", "97.5 %")
rownames(difs) = c("intercepts", "slopes")
difs
#            difference      2.5 %   97.5 %
# intercepts  -61.13285 -186.53497 64.26927
# slopes       13.06666  -18.86799 45.00131