6

This should be a simple inquiry. Doing a regression analysis I found that the coefficient of a predictor has a(n) (infinitesimal) positive effect of 0.001 that is significant at the 0.005 level.

I cannot help wondering how a similar phenomenon may be possibile. Do you think it might depend on the fact that my variables are not (yet) scaled? As soon as I have rescaled all the predictors, do you think that the magnitude of the effect in point would change?

I hope someone is able to clarify this puzzle.

another_newbie
  • 73
  • 4
  • 1
    Thank you all for you enlightening answers! I have just recoded the relevant predictor along the same scale and, as you pointed out, the magnitude of the coefficient rocket up to 0.15xxx with p < 0.001. Super! – another_newbie Sep 16 '20 at 10:01

2 Answers2

16

A simple thought experiment: suppose your predictor was a length, originally expressed in millimetres. If you express it instead in kilometres and fit the model again, you have not really changed anything meaningful about the relationship, but your coefficient will drop by several orders of magnitude.

You can also get significant results with very low coefficients if you have a very large dataset.

mkt
  • 11,770
  • 9
  • 51
  • 125
  • 2
    In fact, after switching to kilometers, the coefficient will drop by exactly six orders of magnitude. – Alexis Sep 15 '20 at 20:14
6

This is a known phenomenon, as p-values depend both on the effect size and the sample size.

As you get many observations, you get convincing evidence that a tiny effect is real.

In other words, that coefficient probably isn’t zero; you have a very unusual result for a situation where the coefficient isn’t zero.

However, the effect isn’t necessarily enough to interest an investigator.

“Yep, it’s not zero, but it’s not big enough to care.”

What you say about scaling might matter. For instance, $0.001$ light years is still a pretty far distance if the other measurements are in centimeters.

Dave
  • 28,473
  • 4
  • 52
  • 104