5

I'm a neuroscientist and I've done some statistics before, but I'm fairly new to corrections of the p-value (like Bonferroni).

I've got 2 groups of patients. For each group, I look into 3 areas of the body. For each area, I've got measurements for 3 variables whose values are not normally distributed.

For each group of patients and area, I try to see whether the numbers obtained for those variables correlate, thus meaning the number of one of them could affect the number of the others in that specific area, in that group of patients. To do this, I perform Kendall tests, which is non-parametric, to assess correlation.

In total, I've got 18 different correlations, and I really don't know whether I need to apply a p-value correction or something similar, because, of course, if you try to correlate many things, you're bound to get a low p-value for some of them.

Ferdi
  • 4,882
  • 7
  • 42
  • 62
Ale
  • 53
  • 4

1 Answers1

2

Yes, you still need to use a multiple correction method. See my answer here for a fuller explanation. Although your tests are non-parametric that will not matter. You are still performing a family of tests - presumably at the same $\alpha$ level and you can expect $18 *\alpha$ false positives by chance alone.

Bonferroni is very conservative, better alternatives exist. if you had a relatively large number of tests then you could use FDR (it requires enough empirical data to determine parameters). However, in this case, with 18 tests you can use an alternative suce as Holm Bonferroni.

martino
  • 1,630
  • 11
  • 17
  • Thank you! Would you say my comparisons are independent from each other? I'm not sure of what this really means. Also, since I'm separating the cohort in two and finding correlations separately in each group, shouldn't I do 2 tests for 9 correlations each? Thanks! – Ale Oct 16 '14 at 14:22
  • Impossible to say really. If the data is dependent then the comparisons would be. It depends on what data you are collecting from the body but it is easy to imagine cases where this data wouldn’t be independent – martino Oct 16 '14 at 14:30
  • 1
    Do you mean two separate multiple test corrections – one for each set of 9 tests. The short answer is no. It doesn’t matter if your tests are dependent or independent. You are carry out a study and performing 18 tests, this is a family of tests and by correcting all 18 as a family you will protect against finding even 1 false positive at the $\alpha$ level. – martino Oct 16 '14 at 14:36