12

I was reading through a paper and I saw a table with a comparison between PPV (Positive Predictive Value) and NPV (Negative Predictive Value). They did some kind of statistical test for them, this is a sketch of the table:

PPV    NPV    p-value
65.9   100    < 0.00001
...

Every rows refers to a particular contingency table.

What kind of hypothesis test did they do? Thanks!

Simone
  • 6,513
  • 2
  • 26
  • 52

2 Answers2

17

Assuming a cross-classification like the one shown below (here, for a screening instrument)

alt text

we can define four measures of screening accuracy and predictive power:

  • Sensitivity (se), a/(a + c), i.e. the probability of the screen providing a positive result given that disease is present;
  • Specificity (sp), d/(b + d), i.e. the probability of the screen providing a negative result given that disease is absent;
  • Positive predictive value (PPV), a/(a+b), i.e. the probability of patients with positive test results who are correctly diagnosed (as positive);
  • Negative predictive value (NPV), d/(c+d), i.e. the probability of patients with negative test results who are correctly diagnosed (as negative).

Each four measures are simple proportions computed from the observed data. A suitable statistical test would thus be a binomial (exact) test, which should be available in most statistical packages, or many online calculators. The tested hypothesis is whether the observed proportions significantly differ from 0.5 or not. I found, however, more interesting to provide confidence intervals rather than a single significance test, since it gives an information about the precision of measurement. Anyway, for reproducing the results you shown, you need to know the total margins of your two-way table (you only gave the PPV and NPV as %).

As an example, suppose that we observe the following data (the CAGE questionnaire is a screening questionnaire for alcohol):

alt text

then in R the PPV would be computed as follows:

> binom.test(99, 142)

    Exact binomial test

data:  99 and 142 
number of successes = 99, number of trials = 142, p-value = 2.958e-06
alternative hypothesis: true probability of success is not equal to 0.5 
95 percent confidence interval:
 0.6145213 0.7714116 
sample estimates:
probability of success 
             0.6971831

If you are using SAS, then you can look at the Usage Note 24170: How can I estimate sensitivity, specificity, positive and negative predictive values, false positive and negative probabilities, and the likelihood ratios?.

To compute confidence intervals, the gaussian approximation, $p \pm 1.96 \times \sqrt{p(1-p)/n}$ (1.96 being the quantile of the standard normal distribution at $p=0.975$ or $1-\alpha/2$ with $\alpha=5$%), is used in practice, especially when the proportions are quite small or large (which is often the case here).

For further reference, you can look at

Newcombe, RG. Two-Sided Confidence Intervals for the Single Proportion: Comparison of Seven Methods. Statistics in Medicine, 17, 857-872 (1998).

chl
  • 50,972
  • 18
  • 205
  • 364
  • Thanks. Ok, I read in very beginning of the paper that they used Chi-square test for all categorical variables. The classification table written doesn't refer to a variable in particular, it is the output of a classification task. It isn't very clear! Now I suppose they did a classical test on proportion.. maybe Chi-square.. – Simone Jan 17 '11 at 14:53
  • I had a look to this question again and I have seen that p-value doesn't refer neither PPV nor NPV, it refers to the entire row. I think the test they had should be associated to the whole contingency-table. – Simone Apr 12 '11 at 13:33
  • @Simone So, if I understand you correctly, you suggest that the authors provide PPV and NPV values but gave the p-value corresponding to a global association test of the 2x2 table? Is it related to this recent question, http://stats.stackexchange.com/questions/9464/can-someone-give-me-details-about-a-particular-confidence-interval-formula? – chl Apr 12 '11 at 18:37
  • Yes, it would be related to that question if p-value were associated either to PPV or to NPV. And in that case you gave the solution. The test corresponds to the whole 2x2 table, I won't ever know what kind of test it is! – Simone Apr 12 '11 at 20:11
1

Please see

Kosinski, Andrzej S. A weighted generalized score statistic for comparison of predictive values of diagnostic tests. Statistics in Medicine http://dx.doi.org/10.1002/sim.5587 published online: 22 AUG 2012

Andrzej Kosinski
  • 11
  • 1