Is Bonferroni correction needed for paired t-test on grouped data?

Question

I have a survey that was conducted in 3 different classes ( math, phys, bio) at the beginning and at the end of the semester ( pre and post). In the survey, there were 3 groups of questions (A, B, C) and a Likert-type scale. I converted all the answer into the numerical score

I have the following dataset with the following columns: Subject: math, physics, bio Survey: pre and post q: A, B, and C Score: values ranging 1-7

I want to test for each course for each question type whether the is a difference in score between pre and post-term survey. I conducted paired t-test but my question is do I need a Bonferroni correction here?

Here is some simulated data and the code for paired t-test:

df= data.frame(
survey = rep(c("pre","post"),60),
subject = rep(c("bio", "math", "phys"),40),
q = rep(c("A", "B", "C"),40),
score =  sample(x=1:7, size = 120, replace = TRUE))
df

df %>%  group_by(subject, q) %>% 
  t_test(score ~ survey, paired = TRUE, detailed = TRUE) %>%
  add_significance()

Answer 1

The short answer: "It Depends".

The longer answer:

Whether to adjust for multiple comparisons is a very good question without a good/standard answer. It really does depend on things other than the data and basic questions.

I like the approach of thinking through how it would change my interpretation of the results if I were to add an additional 20 (or 100 or other large number) of categories/tests/intervals based on random data to my process and either adjust for the multiple comparisons or not adjust.

Think about for your case if you added a bunch more categories and filled in the pre and post measurements with purely random data.

Scenario 1: You plan to declare "Success", publish, change policy, etc. if any of the tests are significant. So adding 20 or more tests on random data without adjusting for multiple comparisons greatly increases your chance of seeing at least one significant test (but adjusting for all the tests does not).

Scenario 2: You are really interested in the "Math" category and whether "Physics" shows a difference or not will not affect your interest in the math class. But adding 20 or more additional tests on random data and adjusting for multiple comparisons will increase the C.I. interval width on math giving less precision.

In scenario 1, definitely adjust. In scenario 2, definitely do not adjust. But most real scenarios fall somewhere in between, so you really need to think about what effect adjusting or not will have on your decisions.

Note that there are other options that Bonferroni. If you are really interested in this then I would look at Bayesian Hierarchical models. Instead of just increasing the standard error because some effect sizes may be too extreme by chance, they actually shrink the extreme effect sizes using information from the other groups giving better effect size estimates (and intervals) as well as allowing for "Partial Correction" instead of just a yes/no.