Coronavirus: False Positives, False Negatives and the Trouble with COVID-19 Mass Testing

We are still in the midst of the COVID-19 lockdown. The conventional wisdom -- which has been wrong several times -- is that the only way to lift the lockdown is through mass testing. Test here. Test now. Test, baby, test!

An article in Vox claims that we might need 35 million tests per day, which means testing people over and over again. Putting aside the logistical nightmare, consider that the U.S. is currently testing roughly 100,000 Americans per day. Asking for 35 million per day is sort of like requesting a golden toilet seat on your fusion powered spacecraft to Mars.

The reality of mass screening is that it's never done. Besides the impracticality, there is a scientific reason for it: Diagnostic tests are imperfect, and they produce false positives and false negatives. Even if kept to a minimum, minor inaccuracies create substantial problems. To understand why, we must consider a diagram that every epidemiology and biostatistics student has seen:

Credit: Quora

In the upper-left corner are True positives (TP), those who have the disease and produce a positive test result. In the lower-right corner are True negatives (TN), those who do not have the disease and produce a negative test result. False positives (FP) and false negatives (FN) are self-explanatory.

The formulas surrounding the box relate to the accuracy or reliability of the diagnostic test. Sensitivity answers the question, "Of all the people who truly have the disease, how many will produce a positive test result?" Specificity answers the question, "Of all the people who truly do not have the disease, how many will produce a negative test result?"

Obviously, a test with 100% sensitivity and specificity is ideal, but we don't live in an ideal world. In reality, there is an inverse relationship between sensitivity and specificity; as one increases, the other decreases. For example, one test for tuberculosis has a sensitivity of 92% and a specificity of 90%.

Let's now apply all this to COVID-19 mass testing. And let's assume a sensitivity of 95% and a specificity of 90%. If we tested 100 individuals who we know for sure have the disease and 100 individuals who we know for sure do not, the numbers would look like this:

The positive predictive value (PPV) answers the question, "If a person receives a positive test result, what is the probability that he actually has the disease?" The negative predictive value (NPV) answers the question, "If a person receives a negative test result, what is the probability that he actually does not have the disease?" In our example, the PPV and NPV are 90% and 95%, respectively.

In other words, if a person receives a positive COVID-19 test result, there is a 10% chance that he doesn't actually have the disease. Similarly, if a person receives a negative COVID-19 test result, there is a 5% chance that he actually does have the disease.

The example we used had 100 known COVID-19 cases and 100 known healthy people. This means that 50% of our test sample had COVID-19. What happens if the prevalence of COVID-19 isn't 50%? If we did a mass screen, it is highly unlikely that 50% of the American public actually has COVID-19. Let's assume, for the sake of argument, that 10% of Americans actually have the disease.

If we test 200 people at random and the actual prevalence of COVID-19 is 10%, then we know that 20 people in our sample should have the disease, while the other 180 should not. What does this do to the PPV and NPV?

The NPV is fantastic. If a patient receives a negative test result, there is a 99% probability that he actually does not have the disease. The PPV isn't so good. If a patient receives a positive test result, there's only a 51% chance that he has the disease. Largely for this reason, public health officials advise against mass disease screens, such as for cancer. For emotional and financial reasons, it's a really bad idea to tell somebody, "You have cancer," only to tell them later, "Oops, sorry. You don't."

But COVID-19 isn't cancer, and there may be relatively little downside in implementing a mass screen. The point is that mass screens are hardly a panacea. Those who insist on mass COVID-19 screening should understand the limitations and implications of that public health policy. My colleague Dr. Chuck Dinerstein and RCP's Sean Trende both have excellent articles exploring this further.