Probabilty Scale Versus Log Odds - Interaction Effect Size

Question

I am looking at doing some power analysis for a simple full factorial design using logistic regression as the method of analysis 2 main effects and their interaction. I am trying to discern the magnitude of the interaction through postulated probabilities.

Specifically I have the following factors A and B, each with a low (-1) and high (1) level. Given a balanced experiment I believe I can average over the other level for the main effects and calculate the main effect of A as 0.11% and the main effect of B as 0.0934%. Slicing the simple effects of A I calculate the interaction as -0.08%.

I am therefore thinking with the power analysis I should have almost as much power for the interaction term as the other main effects, especially B. It works out that the interaction term is tiny through the logistic regression.

Is there any way to tell just from the probabilities if the interaction is "large" or "small"?

> x_df<-data.frame(A=c(-1,-1,1,1),B=c(-1,1,-1,1))
> x_df
   A  B
1 -1 -1
2 -1  1
3  1 -1
4  1  1
> X<-matrix(c(12,9988,5,9995,25,9975,10,9990),nrow = 4,ncol = 2,byrow = TRUE)
> X
     [,1] [,2]
[1,]   12 9988
[2,]    5 9995
[3,]   25 9975
[4,]   10 9990
> summary(glm(X~x_df$A+x_df$B+x_df$A*x_df$B, family = binomial))

Call:
glm(formula = X ~ x_df$A + x_df$B + x_df$A * x_df$B, family = binomial)

Deviance Residuals: 
[1]  0  0  0  0

Coefficients:
              Estimate Std. Error z value Pr(>|z|)    
(Intercept)   -6.80509    0.16274 -41.817  < 2e-16 ***
x_df$A         0.35723    0.16274   2.195  0.02815 *  
x_df$B        -0.44849    0.16274  -2.756  0.00585 ** 
x_df$A:x_df$B -0.01041    0.16274  -0.064  0.94901    
---
Signif. codes:  
0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 1.5995e+01  on 3  degrees of freedom
Residual deviance: 7.0046e-12  on 0  degrees of freedom
AIC: 25.033

Number of Fisher Scoring iterations: 3

Answer 1

There is a general rule of thumb: If you calculated a sample size of $n$ to test for the main effect, you'll need $4n$ for the interaction of the same magnitude and at the same power. See Leon and Heo (CSDA, 2009). This applies to logistic regression as well. It can be worse for poorly balanced data.

Answer 2

I don't really know if you want a power analysis (which would be estimating the probability of having a significant result given that there is a true effect of a certain size, and which is not really a legitimate thing to do based on observed sample statistics).

In terms of power, you probably had low power to detect the real value of the interaction if there is a real interaction effect - as you had a non-significant result - (emphasis on probably, as you may have just gotten very unlucky, there is no actual way of figuring this out without knowing the true population effect). The smaller the effect, the lower the power, all else being equal, so unless you expect that the interaction to be quite a lot larger than it is estimated to be here, you had super low power to detect an effect of that magnitude. It certainly wouldn't be the same as for the main effects.

In fact, it would have had to have been massively overestimated (given that the magnitude here is accurate - not a safe assumption) to be statistically significant in your dataset.

Effect sizes are notoriously difficult to interpret for interaction terms in log reg- the coefficients tell you how much of a change there is in the log odds, not in terms of percentages, and as it's an interaction, it can only be interpreted in the context of the scales of the two independent variables - you may want to have a look at https://stats.stackexchange.com/a/57052/163818 for a primer on how to interpret logistic regression interactions.