4

Let's say I have some sample of people with a measure of interest for each one, let's call it v. Let's say I measure v twice for the same set of people, one in January and one in December. Now I want to assess whether a person's v value improved from January to December. Let's assume that a high v value means they did well.

I thought of the following analyses, and I think I have seen it somewhere, and I wanted to know what the name of this is.

Consider taking each v value in January and giving each person a z-score. That is,

January_z_i = (January_v_i - mean(January_v))/sd(January_v))

where January_z_i is the z-score for a single person, January_v_i is the v value for a single person in January, and January_v represents the entire list of v values for January.

Now for December, I go ahead and do the following

December_z_i = (December_v_i - mean(January_v))/sd(January_v)

Notice how I used the mean and sd of January_v for the December group. The logic is that, assuming the same sample space as measured in January, did that same person improve within that space if you consider their score in December.

What is the technical name for this analysis. Sorry I didn't explain it more succinctly.

user164846
  • 395
  • 2
  • 4
  • 9

2 Answers2

2

I don't think this is the right type of analysis.

Consider this group of 2 persons only, call them $X_1, X_2$. In Jan, they got a score of $(1,99)$ respectively. In Dec, they both got 99. However, using your test: ${X_{1,Dec}-\bar{X}_{Jan}\over sd(X)_{Jan}}\approx 0.7$ which is probably not 'significant' depending on the cutoff you use. But one would think a jump from 1 to 99 is a significant improvement. In fact, all your test is doing is to test whether a December score is significantly different to an average score in Jan.

You should really be using paired tests. For example, if the test scores can be assumed to be Normal, then you can use tests like the paired-t-test. (you can google it). If not, you can try non-parametric tests like the sign-rank test.

But the key term is really paired test.

qoheleth
  • 1,210
  • 1
  • 12
  • 25
1

The answer by qoheleth provides the correct test that you should use (e.g. paired sample t-test). However, your question was what the name of the test is you suggested. In fact, I think, it goes under the name of a "z-test", which is very close to your title.

The point is that you assume january responses are samples from a reference 'population' and then you test for each individual, and the sample mean, at the second occasion in December, whether they are samples from the January population. This is the standard textbook test when introducing z-scores, because it gives you a p-value for each individual at the second occasion under the assumption that observations at the first occasion in January are samples from a normal distribution.

tomka
  • 5,874
  • 3
  • 30
  • 71