Variable becomes insignificant after including variable in multiple regression

Question

I am fairly new to this, sorry if I am not clear.

I have two models with logY. The first model has a slightly bigger sample than the second model and accounts for 4 main drivers (and some control variables). In the first model:

X1 = significant
X2 = significant
X3 = significant
X4 = insignificant

The second model as 2 more variables (same control variables) and the sample is slightly smaller:

X1 = significant
X2 = significant
X3 = INSIGNIFICANT
X4 = insignificant
X5 = insignificant
X6 = significant

Thus after including the two variables, X3 becomes insignificant in the second model.

First, I checked if it had something to do with the sample, so I run the regression of the first model on the same sample as the second model. X3 didn't become insignificant. Thus I included X5 and X6 one by one and found that when I include X6 my X3 becomes insignificant. Therefore I assumed that there might be some correlation problems and checked for that, but the correlation between X6 and X3 is "only" -0.5. If it is not a correlation problem, then what can it be?

All my X are 0/1 dummy variables & I am using STATA (new to it)

Answer 1

Remember that the $t$ tests for linear regression test hypotheses of the form $$H_{0,j} : \beta_j = 0 \big\vert \beta_{-j} \hspace{1cm} \textrm{ vs. } \hspace{1cm} H_{1,j} : \textrm{ not } H_{0, j}.$$

Let's say we add a predictor $X_{p+1}$ and all of a sudden $X_1$ goes from significant to insignificant. The most likely explanation is that $X_{p+1}$ explains $X_1$ quite well, so after accounting for the effect of $X_{p+1}$ $X_1$ contributes nothing of value. So this is a direct result of correlation between predictors, and you've seen that you do have correlation between your predictors. The standard diagnostics for investigating this are variance inflation factors and looking at the eigenvalues of $X^T X$. This will be much more informative than computing pairwise correlations.

Note that if your $X$ matrix is orthogonal then you can add and delete predictors (assuming orthogonality is preserved) without affecting any of the other $\beta_j$s.

Update

Consider the following example:

set.seed(123)
n <- 100
x0 <- rnorm(n)
x1 <- rnorm(n)
x3 <- rnorm(n)
eps <- rnorm(n)

x2 <- x3 + rnorm(n, 0, 2)

print(cor(x2,x3))
# 0.4144069  # less than your example of 0.5

dat <- data.frame(x0=x0, x1=x1, x2=x2, x3=x3)

y <- 1 + x0 + x1 + x3 + eps

mod_no3 <- lm(y ~ x0 + x1 + x2, data = dat)
mod_3 <- lm(lm(y ~ x0 + x1 + x2 + x3, data = dat))

print(summary(mod_no3))  # p-val for x2 is 0.01, definitely significant
print(summary(mod_3))  # p-val for x2 is .8, not even a little significant

car::vif(mod_3)  # none of these are even 1.5
#       x0       x1       x2       x3 
# 1.064094 1.023699 1.282272 1.216728