Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [post-hoc]

"post-hoc" refers to analyses that are decided upon after the data has been collected, as opposed to "a priori".

"post-hoc" refers to analyses that are decided upon after the data has been collected, as opposed to "a priori". The more the data is examined post-hoc and the more statistical tests are run on it, this should be taken into account by multiple-testing procedures.

It is also a (outdated) concept of causality "post hoc ergo propter hoc" (meaning "after this, therefore because of this"). The idea is that if we always observe event $Y$ after event $X$, $X$ must have caused $Y$. This concept underlies for instance the Granger causality test. Nowadays this post hoc notion of causality is not accepted as actual causality.

598 questions
64
votes
2 answers

Do we need a global test before post hoc tests?

I often hear that post hoc tests after an ANOVA can only be used if the ANOVA itself was significant. However, post hoc tests adjust $p$-values to keep the global type I error rate at 5%, don't they? So why do we need the global test first? If…
even
  • 2,147
  • 6
  • 18
  • 13
43
votes
2 answers

What are the practical differences between the Benjamini & Hochberg (1995) and the Benjamini & Yekutieli (2001) false discovery rate procedures?

My statistics program implements both the Benjamini & Hochberg (1995) and Benjamini & Yekutieli (2001) false discovery rate (FDR) procedures. I have done my best to read through the later paper, but it is quite mathematically dense and I am not…
russellpierce
  • 17,079
  • 16
  • 67
  • 98
32
votes
3 answers

Can ANOVA be significant when none of the pairwise t-tests is?

Is it possible for one-way (with $N>2$ groups, or "levels") ANOVA to report a significant difference when none of the $N(N-1)/2$ pairwise t-tests does? In this answer @whuber wrote: It is well known that a global ANOVA F test can detect a…
amoeba
  • 93,463
  • 28
  • 275
  • 317
26
votes
3 answers

Post hoc test after ANOVA with repeated measures using R

I have performed a repeated measures ANOVA in R, as follows: aov_velocity = aov(Velocity ~ Material + Error(Subject/(Material)), data=scrd) summary(aov_velocity) What syntax in R can be used to perform a post hoc test after an ANOVA with repeated…
L_T
  • 1,463
  • 7
  • 20
  • 25
23
votes
1 answer

Bonferroni or Tukey? When does the number of comparisons become large?

Reading Field's Discovering Statistics Using SPSS (3rd Edition) I was struck by a bit about post-hoc tests in ANOVA. For those wanting to control the Type I error rate he suggests Bonferroni or Tukey and says (p. 374): Bonferroni has more power…
Parbury
  • 317
  • 1
  • 2
  • 8
23
votes
4 answers

Is Fisher's LSD as bad as they say it is?

When we perform experiments (on small sample sizes (usually the sample size per treatment group is about 7~8)) on two groups, we use a t-test to test for difference. However, when we perform an ANOVA (obviously for more than two groups), we use…
Rover Eye
  • 535
  • 1
  • 4
  • 15
22
votes
1 answer

Post-hoc tests after Kruskal-Wallis: Dunn's test or Bonferroni corrected Mann-Whitney tests?

I have some non-Gaussian distributed variable and I need to check if there are significant differences between the values of this variable in 5 different groups. I have performed Kruskal-Wallis one-way analysis of variance (which came up…
Rossella
  • 471
  • 1
  • 3
  • 8
22
votes
3 answers

How to perform post-hoc test on lmer model?

This is my data frame: Group <-…
Ping Tang
  • 425
  • 1
  • 5
  • 7
21
votes
3 answers

How can I get a significant overall ANOVA but no significant pairwise differences with Tukey's procedure?

I performed with R an ANOVA and I got significant differences. However when checking which pairs were significantly different using the Tukey's procedure I did not get any of them. How can this be possible? Here is the code: fit5_snow<- lm(Response…
L_T
  • 1,463
  • 7
  • 20
  • 25
20
votes
1 answer

Why use ANOVA at all instead of jumping straight into post-hoc or planned comparisons tests?

Looking at a between-groups ANOVA situation, what do you get by actually doing such an ANOVA test first, and second do post-hoc (Bonferroni, Šidák, etc.) or planned comparisons tests? Why not skip the ANOVA step entirely? I gather that in such a…
Sympa
  • 6,862
  • 3
  • 30
  • 56
20
votes
5 answers

Post-hocs for within subjects tests?

What is the preferred method for for conducting post-hocs for within subjects tests? I've seen published work where Tukey's HSD is employed but a review of Keppel and Maxwell & Delaney suggests that the likely violation of sphericity in these…
russellpierce
  • 17,079
  • 16
  • 67
  • 98
18
votes
2 answers

How do you write up Tukey post-hoc findings?

What is the proper way to write a Tukey post-hoc result? There are several examples with different results? Say you have North, South, East and West. North N=50 Mean=2.45 SD=3.9 std error=.577 LB=1.29 UB=3.62 South N=40 Mean=2.54 SD=3.8 std…
Sue
  • 181
  • 1
  • 1
  • 4
17
votes
2 answers

Which multiple comparison method to use for a lmer model: lsmeans or glht?

I'm analyzing a data set using a mixed effects model with one fixed effect (condition) and two random effects (participant due to the within subject design and pair). The model was generated with the lme4 package:…
schvaba986
  • 273
  • 1
  • 2
  • 7
17
votes
2 answers

Post-hoc test for chi-square goodness-of-fit test

I'm conducting a chi-square goodness-of-fit (GOF) test with three categories and specifically want to test the null that the population proportions in each category are equal (i.e., the proportion is 1/3 in each group):                 OBSERVED…
Meg
  • 1,633
  • 4
  • 18
  • 28
16
votes
6 answers

What is the problem with post-hoc testing?

My statistic professor says so, all the books that I look at state it: post-hoc-testing is unscientific. You must derive a hypothesis from theory first, and then collect data and analyse it. But I really don't understand what the problem…
user14650
1
2 3
…
39 40