3

I have paired data as follows. These are proportion correct scores (ranging from 0 to 1) from multiple experiments in which Treatment X and Treatment Y were administered.

Treatment X: 0.25, 0.35, 0.15, 0.15, etc. Treatment Y: 0.55, 0.56, 0.33, 0.08, etc.

Based on theoretical predictions, I should expect close to a 1:1 relationship between data from Treatment X and Treatment Y. So for instance, experiment 1, the scores should be the same (but they're not, 0.25 vs. 0.55).

Accordingly, if I create a scatterplot of X vs. Y scores, and the 1:1 relationship held, all points should be on a diagonal line with slope 1,1 and intercept at 0.

The key question is: how do I test for Treatment X being different from Treatment Y (or not)? (My data indicate that predictions in the literature are wrong, and there isn't a 1:1 relationship; I'd like to find a way of testing that). Two questions follow:

  • My first thought is to do a sign test, 2-tailed. In R, however, binom.test requires positive integers. Is there a workaround for that?
  • Alternatively, do any of you suggest a more sophisticated way to test the relationship between Treatment X and Treatment Y? For instance, a way to assess how many points fall above or below the predicted relationship (e.g., Y higher than X, or X higher than Y)?

Thanks for any suggestions!

Altair555
  • 61
  • 2

2 Answers2

1

The thing that the sign test counts is the number of pairs for which the first variable exceeds the second (or vice-versa).

(The values of the observations are supposed to be continuous rather than discrete, so that that under the null, $p=0.5$).

So the 'workaround' is simply to supply binom.test with the correct information.

[If we exclude ties, then something like binom.test(sum(x>y),length(x!=y)) should do]

Glen_b
  • 257,508
  • 32
  • 553
  • 939
  • Gee, you are on the ball. +1 for all the help you give, although I like my own answer better. – Carl Nov 15 '16 at 01:20
  • Thank you Glen_b for these clarifications and the correct code! Wonderful, and gratefully appreciated! – Altair555 Nov 15 '16 at 03:23
1

Since the data are paired, the sign test computes the probability of obtaining a total number of + or - results from the difference between values. The sign test is easy to understand but not always very powerful. The Wilcoxon signed-rank test may be a better choice as it can be more powerful for discriminating between the location of paired data of this type.

Carl
  • 11,532
  • 7
  • 45
  • 102
  • 1
    +1 The signed rank test assumes that the distribution of pair differences will be symmetric when the null is true (it doesn't have to be the case under the alternative). Since people are often prepared to make the somewhat stronger assumption that under the null the two variables have the same distribution (within each pair), this isn't usually controversial; however sometimes it is not reasonably clear that it should be the case even under the null, and then the sign test may be better. [Also, if the distribution is quite peaky/heavy-tailed the sign test may have more power even so] – Glen_b Nov 15 '16 at 01:33
  • @Glen_b Answer amended to reflect your comments. – Carl Nov 15 '16 at 01:43
  • 1
    I'll leave the comment there for the additional detail it provides. – Glen_b Nov 15 '16 at 01:44