Scenario: comparison of 2 different binary classifiers
Both classifiers report sensitivity and specificity and number actually positive (P), but classifier 1 is tested on a dataset with prevalence 20%, classifier 2 is tested on an enriched sample with prevalence 50%. I prefer PPV to specificity in my context, because I'm much more interested in the classification of the positive class, and can plot both classifiers' results on precision recall curve (PRC) space rather than ROC space, however classifier 2 will have a higher PPV (precision) just by virtue of having a higher prevalence.
Is it valid then to say, provided that sensitivity and specificity of both classifiers do not change, then for prevalence of 10%, I can calculate a prevalence-adjusted PPV (keeping the ratios of TP/FN and TN/FP the same)?
