For a coefficient in a regression model, yes, if the $1-\alpha$ interval doesn't contain the value of the parameter under the null hypothesis, that same null value would be rejected by a test at significance level $\alpha$.
Indeed, both procedures are based on the same pivotal quantity
In the case of the hypothesis test, the statistic is $T=\frac{\hat{\beta}-\beta_0}{s_\hat{\beta}}$, which is rejected when $|T|\geq t_{1-\alpha/2}$ (at the relevant degrees of freedom; n-2 for simple linear regression).
The $1-\alpha$ confidence interval has bounds of $\hat\beta\pm s_\hat{\beta}\cdot t_{1-\alpha/2}$.
If $\beta_0$ is outside the interval $\hat\beta\pm s_\hat{\beta}\cdot t_{1-\alpha/2}$, that's equivalent to saying $|\hat{\beta}-\beta_0|\geq {s_\hat{\beta}}\cdot t_{1-\alpha/2}$.
Now let's show that rejecting in the hypothesis test is the same:
$|T|\geq t_{1-\alpha/2}\implies |\frac{\hat{\beta}-\beta_0}{s_\hat{\beta}}|\geq t_{1-\alpha/2}$
$\hspace{2.55cm}\implies|\hat{\beta}-\beta_0|\geq s_\hat{\beta}\cdot t_{1-\alpha/2}$
(Not all tests have this property, but many do. There's an explicit example of one that doesn't in comments here)
Note that both inferential procedures rely on the same set of assumptions; if the assumptions don't hold (or reasonably nearly so), the actual type I error rate and the actual coverage probability might not be very close to their nominal values.
