2

There are two Covid-19 tests

  • Test A: Sensitivity 92%, Specificity 99.7%
  • Test B: Sensitivity 97%, Specificity 99.7%

Assume a person takes three tests consecutively (i.e. right after another), in the order A, A, B.

The results are as follows:

  • First time (using Test A): Positive
  • Second time (using Test A): Positive
  • Third time (using Test B): Negative

I've tried to calculate the probability that person is actually positive, but I'm making a mistake somewhere.

Can someone tell me how to calculate this? I'm not sure if additional information would be needed, such as the Covid-19 incidence?

Make any simplifiying assumptions you need, and assume the test results are independent of each other.

ChrisC
  • 123
  • 3
  • 2
    It cannot be calculated without making strong simplifying assumptions, which likely are false. In particular, when one kind of test gives a mistaken result for a particular patient, perhaps it will *consistently* produce the same error on retests of the same patient. It is possible the errors made by two different tests could be correlated (either positively or negatively), too. – whuber Jan 17 '22 at 18:19
  • 1
    Ok, understood :). I edited the question to say that any simplifying assumptions should be made, and tests should be assumed as independent. Does that work? – ChrisC Jan 17 '22 at 18:22
  • 2
    Yes it does, thank you. (You evidently also need to assume the patient's status is the same for all three tests; that is, either they were definitely infected or definitely not infected at the times each of the tests were taken.) – whuber Jan 17 '22 at 18:29
  • Ah ok, that’s what I thought. Not sure what would be a sensible (of course grossly simplified, but somewhat sensible under simplified assumptions) probability. This is for Germany. In the past 7 days an approximate average of 810.000 people have had ongoing (i.e. „active“ infections) according to statista https://de.statista.com/statistik/daten/studie/1181971/umfrage/aktive-faelle-des-coronavirus-in-deutschland/. Let’s assume that is accurate (which it might not be), would it be somewhat sensible to divide that average of 810.000 active infections by Germany‘s population of… – ChrisC Jan 17 '22 at 18:50
  • … about 83.24 million to arrive at probability of 0.97% that she is actually infected, and use that for the calculation in question? Actually not sure if that would be biting the cat in the tail, since I’m trying to calculate a probability from tests and now I calculated a probability from active infections/population, to use that in the next probability calculation ^^ – ChrisC Jan 17 '22 at 18:53
  • And yes, assume her actual status was the same during all three tests, as they were taken within minutes of each other. – ChrisC Jan 17 '22 at 18:55
  • Btw, my musings about dividing active infections by population came about after I misread your comment. Initially I read that you had said her *actual* status has to be known for the calculation. But you were just saying her actual status would have to be assumed constant across all three tests, which is a valid assumption for this case. – ChrisC Jan 17 '22 at 18:58
  • 1
    You do need the ratio of infected to total population, see answer below. – ecnmetrician Jan 17 '22 at 19:15

1 Answers1

1

Notation:

The results of the tests are $T_1,T_2,T_3 \in \{0,1\}$. We indicate a positive test by $T_i = 1$ and zero otherwise, for $i \in \{1,2,3\}$. Similarly, suppose that $Y$ is the status of having covid $(Y=1)$ or not $(Y=0)$.

The accuracy of each test is given by:

  • The sensitivity, $\alpha_i = \mathbb{P}(T_i = 1 \mid Y=1)$
  • The specificity, $\beta_i = \mathbb{P}(T_i = 0 \mid Y = 0)$.

Problem:

We want to compute the conditional likelihood of infection $(p)$, which is equal to $$ p = \mathbb{P}(Y=1 \mid T_1 = 1, T_2 = 1, T_3 = 0) $$ For ease of notation let $\mathcal{T}$ denote the event that the tests have two positives and one negative. Then $$ p = \mathbb{P}(Y = 1 \mid \mathcal{T}). $$

A Bayes' approach:

We can write the probability as: $$ p = \frac{\mathbb{P}(\mathcal{T} \mid Y = 1)\mathbb{P}(Y = 1)}{\mathbb{P}(\mathcal{T})} $$ The value $\mathbb{P}(Y = 1)$ is the unconditional probability that somebody has COVID, and $\mathbb{P}(\mathcal{T})$ is the probability of observing a set of testing results in the population. The value $\mathbb{P}(\mathcal{T} \mid Y = 1)$ is the likelihood of a observing a test sequence for an infected person.

We need assumptions to figure out each quantity:

  • Assumption 1: $\mathbb{P}(Y = 1)$ is equal to the local covid infection rate.
  • Assumption 2: The tests $T_1,T_2,T_3$ are independent given $Y$ (the true status)

$$ \mathbb{P}(T_1 = t_1,T_2 = t_2, T_3 = t_3 \mid Y =y) = \prod_{i=1}^3 \mathbb{P}(T_i = t_i \mid Y = y). $$

where $(t_1,t_2,t_3)$ are the values of the test for a specific patient.

Sketch of the computation steps:

First you need to determine the local infection rate (which is not given by your problem). Then you need use the Bayes' formula for $p$.

  • You can compute $\mathbb{P}(\mathcal{T} \mid Y = 1)$ by using Assumption 2, and using the appropriate value of sensitivity and specificity for each test. For example, for the third test you need to compute $1-\alpha_3$ to obtain the likelihood of a negative test.
  • You can compute $\mathbb{P}(\mathcal{T})$ by the relationship: $$ \mathbb{P}(\mathcal{T}) = \mathbb{P}(\mathcal{T} \mid Y = 1)\mathbb{P}(Y = 1) + \mathbb{P}(\mathcal{T} \mid Y = 0)\mathbb{P}(Y=0)$$ In Step 2 you have to be careful keeping track of which probabilities you need, and how you can obtain them from $\{\alpha_i,\beta_i)\}_{i=1}^{3}$. Plugging in the values of $(\alpha_i,\beta_i)$ without thinking about the specific sequence of test results in your problem is a potential pitfall.
ecnmetrician
  • 602
  • 3
  • 8
  • Cool. I'm trying to do the calculations right now, but am encountering some issues. Specifically, I am trying to calculate P(T3 = 0). Here is my process so far: P(T3 = 0) = P(T3 = Negative) = P(T3 = False Negative) + P(T3 = True Negative) = (1-Sensitivity of T3) + Specifity of T3. Plugging in the numbers, the leftmost side of this equation gives, using the values for sensitivity and specifity for Test B: (1-0.97) + 0.997 = 1.027. Not sure if I'm on the wrong track completely, but a probablity larger 1 strikes me as odd :) Edit: think I found my mistake, trying now – ChrisC Jan 17 '22 at 22:42
  • I believe my mistake in my previous comment is trying to calculate, for example, P(T3 = False Negative) as 1- Sensitivity of T3. It would probably be [1- (Sensitivity of T3) - Specifity of T3 - (1- Specifity of T3)]? But I'm not sure if I can work that out directly. Maybe that's why you suggested using Bayes' Formula, which I have no experience with. Might you guide me in solving this numerically? :) – ChrisC Jan 17 '22 at 22:49
  • 1
    In your first comment you need to weigh each probability by $\mathbb{P}(Y=1)$ and $\mathbb{P}(Y= 0)$, respectively. This ensures that the result is always in $[0,1]$. This is known as the law of total probability. Intuitively, it says that you need to weight the false negatives and true negatives by the likelihood that you have covid to begin with, https://en.wikipedia.org/wiki/Law_of_total_probability. – ecnmetrician Jan 17 '22 at 22:59
  • 1
    I encourage you to work through the problem in the solution because all the formulas to solve it are there, no need to bring other knowledge. I used Bayes' formula to produce the equation for $p$, but beyond that it's all self-contained. – ecnmetrician Jan 17 '22 at 23:01
  • 1
    For part 1 of the computation, you need to multiply together the probabilities of observing each test (assuming that the patient has covid: use only the sensitivity parameters). Before doing part 2 you need to do the same thing assuming the patient doesn't have covid. To do part 2, use the law of total probability as indicated in my comments above. – ecnmetrician Jan 17 '22 at 23:03
  • Bayes' rule is a very simple ratio: (Likelihood of observing two positives and one negative if the patient has COVID) / (Likelihood of observing two positives and one negative for all patients). – ecnmetrician Jan 17 '22 at 23:10
  • I’ll give it a shot tomorrow, thanks! :) – ChrisC Jan 17 '22 at 23:11
  • Aha! I talked to a friend today, who was able to give me the same answer as you, and then it made click! I’ll accept this as the correct answer, thanks! :) – ChrisC Jan 18 '22 at 15:35