1

If one has to perform correlation matrix, correction for multiple testing is likely to be useful. If significant P value is taken as (0.05/number_of_variables) (e.g. for 50 variables, 0.05/50 = 0.001), what will be the power or adequacy of such a correction? Can this adequacy be mathematically calculated? Let's assume all variables are normally distributed.

rnso
  • 8,893
  • 14
  • 50
  • 94
  • If there are 50 variables, then there aren't 50 multiple tests, there are 1225 multiple tests. – Penguin_Knight Apr 07 '15 at 12:34
  • I know that (50*50/2). But I want to know if P=0.05/number_of_variables will be strong enough to keep most random correlations out. What would be your opinion. Can this be proved or disproved mathematically? – rnso Apr 07 '15 at 12:36
  • "If one has to perform correlation matrix" -- this formulation is not clear at all. Do you mean "If one has to test for significance each correlation in a correlation matrix"? – amoeba Apr 07 '15 at 12:40
  • [This thread](http://stats.stackexchange.com/questions/2819/correcting-p-values-for-multiple-tests-where-tests-are-correlated-genetics) may be useful? – Penguin_Knight Apr 07 '15 at 12:42
  • @amoeba : I simply want to know the P value significance level to be used for a correlation matrix (for assessing significance of each correlation in that matrix). I wonder if 0.05/number_of_variables is a reasonable value. – rnso Apr 07 '15 at 12:46
  • Reasonable for what? Do you want to control family-wise error rate? – amoeba Apr 07 '15 at 12:57
  • 1
    The benchmark provided by a null hypothesis that all correlations are zero is even remoter from science than a hypothesis that one such is. I've looked at lots of correlation matrices and never imagined that this could be **useful**! Unless your sample size is very small, in which case the correlation matrix is not worth much, selecting according to strength is more useful than selecting according to significance. Depends on your field, however: in physics correlations $<0.9$ in absolute value are a sign of failure; in some other sciences those $>0.9$ are considered too strong to be genuine. – Nick Cox Apr 07 '15 at 13:19
  • @amoeba : By 'reasonable' I mean equivalent to power or usefulness of P=0.05 for single correlation assessment. – rnso Apr 07 '15 at 13:54

0 Answers0