2

I took an antigen test yesterday and was trying to create a tool to help me interpret the results via Bayes' Theorem. I used actual sensitivity and specificity numbers from the CDC, calculated odds of test results given presence of condition, true/false positive/negative rates, and finally presence of disease given test result. One of the posterior odds is greater than 1, which doesn't make sense to me. Can anyone help me understand if this is a valid result or if I've made a mistake in my calculations or methodology? I'm going to try attaching my excel document - please refer to the "duplicate" tab.

enter image description here

Best,

Scott

  • 4
    Odds must be non-negative but can be greater than or less than $1$. Log-odds can take any real value (and possibly $\pm \infty$). Probabilities must be in $[0,1]$ – Henry Dec 09 '21 at 18:17
  • 1
    See https://discourse.datamethods.org/t/determining-post-test-probability-of-covid-19 – Frank Harrell Dec 22 '21 at 14:03

1 Answers1

2

In your table you may have mislabeled a few items since the operating characteristics do not appear to sum to 1 for a given COVID status, nor do the posterior probabilites sum to 1 for a given test result.

I utilized the true positive rate, true negative rate, and what looked like the population prevalence in your screen shot to construct the table below. Let me know if I incorrectly interpreted any of the numbers. For a positive test result I have provided both a confidence level based on the operating characteristics of the test as well as the Bayesian credible level incorporating the population prevalence.

Operating Characteristics No COVID COVID
Negative Test Result 0.98 0.23
Positive Test Result 0.02 0.77
Prior Probabilities 0.95 0.05
Posterior Probabilities
Negative Test Result 0.99 0.01
Positive Test Result 0.33 0.67

Frequentist Confidence Level

The operating characteristics of the screening test are long-run probabilities that are verifiable within a margin of error through repeated testing. Based on these operating characteristics, if the test returns a Positive result we can "rule out" the hypothesis $H_0$: No COVID at the $\alpha=0.02$ level and are therefore $100(1-0.02)\%=98\%$ confident the subject has COVID. Either the subject has No COVID and we have witnessed a $2\%$ event or the subject indeed has COVID. If the subject truly has COVID we would correctly "rule out" $H_0$: No COVID $77\%$ of the time in repeated testing (power). Understanding these long-run probabilities provides the experimenter confidence when inferring the COVID status of the subject at hand based on a single test result.

Bayesian Credible Level

Based on the prior distribution, if the test returns a positive result the posterior probability of COVID is

$$ \frac{0.05\times0.77}{0.05\times0.77 + 0.95\times0.02}=0.67$$

The posterior odds of COVID would then be $0.67/0.33=2.03$. If we have verifiable knowledge that a given subject was randomly selected from an irreducible population where $5\%$ of the members have COVID, then the posterior depicts the long-run probability of COVID status among randomly selected subjects given a particular test result. This long-run probability can be used to make inference on the COVID status of the subject at hand by imagining the subject was instead randomly selected from the posterior distribution. This is a direct contradiction to the earlier claim that the subject at hand was randomly selected from the prior. The posterior sampling frame is correct only if the prior sampling frame is correct, yet there can only be a single sampling frame from which we obtained the randomly selected subject under investigation. In practice we do not have verifiable knowledge of an irreducible prior population from which a subject was selected. One could instead interpret the prior and posterior probabilities as measuring the subjective belief of the experimenter rather than a verifiable long-run probability of randomly selected subjects. This belief is not a verifiable statement about the actual parameter (COVID status of the subject at hand), the hypothesis, nor the experiment (screening test).

Geoffrey Johnson
  • 2,460
  • 3
  • 12