0

I use the awesome package multcomp and emmeans in R. These are the places I heard about the "mvt" method for the first time. And, however, it is what it's actually done in the Dunnett or Tukey method (just different contrasts, all-vs-control and all-pairwise), I'm wondering if this is recognized widely? If I have, say, 3 custom contrasts and want to use the mvt method, will this be rather questioned and force me to use Bonferroni method, only because the latter is commonly used? Let's assume I work using parametric method, so Bonferroni - which doesn't rely on any distributional assumptions, just adjusting the significance level - isn't my only way to go. Have you tried to use this method in your scientific work and was it accepted?

GibbsSampler10
  • 93
  • 7

1 Answers1

1

At least in clinical trials this type of thing is commonly done, even if I'm not sure to what extent the exact package has been used (I've certainly seen emmeans used even in confirmatory clinical trials, but not the particular multiplicity adjustment option). E.g. the multiple contrast tests in the MCP-Mod method (and its various extensions such as confirmatory MCP-Mod) is essentially based on very similar things in the background and widely used. Similarly, various other ways of setting up custom testing procedures are commonly used. Unless your particular field is extremely conservative in its methods and reviewers are not very statistically sophisticated, I would not expect major problems, but you know your field better.

Björn
  • 21,227
  • 2
  • 26
  • 65
  • Thank you very much for your answer. I also saw this in medical and pharmaceutical research. I also guess SAS has a similar option for the LSMEANS statement, let me quote: "The SIMULATE adjustment computes the adjusted p-values from the simulated distribution of the maximum or maximum absolute value of a multivariate t random vector." It seems that's exactly this approach. So it seems this method isn't a "rare, weird method that nobody is using". I'm asking, because everywhere I can see Bonferroni, Holm, but very rarely - the "mvt" method. Thanks! – GibbsSampler10 Aug 26 '20 at 11:24
  • You were right. I found a number of statistical analysis plans, issued by famous pharmaceutical companies, available publicly, mentioning this method. For instance: https://clinicaltrials.gov/ProvidedDocs/41/NCT03320941/SAP_001.pdf – GibbsSampler10 Aug 26 '20 at 13:22
  • 1
    The methods have solid theoretical underpinnings, supported by multiple papers in JASA and elsewhere, eg, Westfall, P.H. and Tobias, R.D. (2007). Multiple Testing of General Contrasts: Truncated Closure and the Extended Shaffer-Royen Method, Journal of the American Statistical Association 102: 487–494. Also: Westfall, P.H. (1997). Multiple Testing of General Contrasts Using Logical Constraints and Correlations, Journal of the American Statistical Association 92, 299–306. – BigBendRegion Sep 12 '20 at 20:38