0

This is a pretest-posttest design with a control group (no treatment between measurements) and an experimental group (treatment between measurements).

All measurements consist of the same rubric and provide ordinal data (1,2,3,4).

Tests for metric data like t-tests are not appropriate. Many websites suggest using the Mann-Whitney U test as a non-parametric test for independent groups, with the difference between pretest and posttest score, i.e, gain score. My problem with these suggestions is that with ordinal data calculations like differences are not possible.

What test would be appropriate for this pre-test/post-test design?

Edit 2: Calculating differences is not possible for ordinal data

Pitouille write about "calculate the difference between post and pre-test". In my understanding, it is not possible to determine the difference with ordinal data, as it is not defined at all. If I could calculate the difference, the data would have to be at least interval scaled.

The difference corresponds to the usual gain score, which I thought of first. But because of the above reasoning, it is not possible to form the gain score.

Hence my question is: How can I determine the effects in pretest-posttest groups if I cannot calculate differences?

Edit 1: Possible examples of data.

First to be mentioned, the study is in the planning phase and I have not yet collected any data. Each group is expected to have 15 to 20 participants.

Candidate Control(0) or Experiment(1) group? Pretest-Score Posttest-Score
A 0 1 1
B 0 1 2
C 0 2 2
D 1 2 3
E 1 2 2
F 1 3 4
Svana
  • 1
  • 1
  • To understand better your problem, it would be helpful if you can share examples of the data you have to deal with. – Pitouille Sep 03 '21 at 05:08
  • @Pitouille Thank you for your quick reply! If the examples attached are not sufficient, please let me know. – Svana Sep 03 '21 at 07:27
  • I feel your question is more about structuring your test... did I get it right? – Pitouille Sep 03 '21 at 12:48

1 Answers1

0

I might miss the point... but at least it could contribute to push your question forward!

My understanding is that you are looking for the approach to conduct your test rather the test itself.

Supposing that the assumptions to conduct a Mann-Whitney U test are met, the approach would be to split your data in 2 samples (control and experiment) and for each candidate, calculate the difference between post and pre test.

Control

Canditate Post - Pre
A 0
B 1
C 0

Experiment

Canditate Post - Pre
D 1
E 0
F 1

... and then perform your test with you two independent samples.

EDIT:

So, I did miss the point :-)

There is study from Winter and Dodou that you can find in the stats literature that could be relevant to your problem: https://blog.minitab.com/en/adventures-in-statistics-2/best-way-to-analyze-likert-item-data-two-sample-t-test-versus-mann-whitney

You can also have a look at @Glen_b's answer: Mann Whitney or two tailed t-test

Pitouille
  • 1,506
  • 3
  • 5
  • 16
  • Formerly, that was my idea too. In the post with the question, I added my concern as it was too long for the comment. – Svana Sep 03 '21 at 16:52