New York City mayor Bill de Blasio had a major problem on his hands last month, one of his own making. He’d promised the city teachers’ union he would shut down the city’s enormous public school system, reopened not so long before, if the city’s “test positivity rate” hit 3 percent. And it had, so he shut the schools. Just 10 days later, with the rate even higher, at 3.9 percent, de Blasio reversed his decision, explaining to CNN that when the 3 percent cutoff that he had once vigorously defended was put into place, the city “did not have the information we have now.”
This move away from the test positivity metric—usually calculated as the number of Covid-19 tests that come out positive divided by the number of tests done—is significant. Since the start of the pandemic, most reporting on the spread of the disease has led with a simpler number: the number of diagnosed cases. This continues to be the top-line stat on major news sites’ Covid trackers, despite the fact that total case numbers have, at times, been quite misleading. Last spring, for example, tests were very scarce, and many cases went undetected. So policymakers came to rely on test positivity as an alternative. Thus the 3-percent threshold for school closures in New York City; or Connecticut’s directive that visitors from states where the test positivity is higher than 10 percent must self-quarantine. But this replacement metric has been misunderstood.
Think about it. Test positivity is not a direct measure of new infections appearing in a population. It’s a ratio, and a ratio increases in two ways: Either when the numerator (in this case, the number of positive tests) rises; or when the denominator (in this case the number of tests performed) falls. Since the number of tests varies from place to place and over time, the test positivity may go up or down even when there is no change at all in disease spread.
As a result, while test positivity may be more informative than raw case numbers, it brings along its own distortions. The ratio will vary with the availability of tests, who’s deciding to get tested, and whether they can get into test centers if they go. The numbers may differ in sub-populations, too. It was still just 0.3 percent in New York City’s schools, for example, when the city’s overall rate jumped to 3.9 percent. And some states offer free testing only to people with symptoms, a policy that is guaranteed to make the test positivity rate go higher.
One reason test positivity ratios came to be so widely used is that they showed up on the dashboard of Johns Hopkins University’s Coronavirus Resource Center early in the pandemic. But they were not meant to be used as a direct measure of coronavirus spread, says Jennifer Nuzzo, the lead epidemiologist at the center. Rather, the numbers were meant to show whether enough testing was being done. When she and her colleagues were first developing the website, they noticed that testing rates varied widely from one country to another. But countries that were doing well in handling the virus and had comprehensive surveillance in place showed test positivity rates between about 3 and 5 percent. “That prompted the realization that it made sense to track testing in that way,” she says. And practically speaking, they had very few other data points to consider at that stage.
Many epidemiologists see policymakers’ reliance on the test positivity ratio in similar terms, as an example of expediency: The number was around, so people started using it. The media, too, has picked up on test-positivity ratios and made them into screaming headlines. “Somewhere along the line, some wires got crossed,” says Michael Mina, a virologist and epidemiologist at the Harvard T.H. Chan School of Public Health. The ratio does not tell you, by itself, how widespread Covid infections have become in your community. “Test positivity is really not reflective of anything unless you know very well who is getting tested, and why,” Mina says.
In October, for example, an ultra-Orthodox Jewish community in New York’s Orange County had a very high positivity rate—34 percent—and the state issued a lockdown. Within a couple of weeks, the rate dropped to 2 percent, and the state opened things up. The county health commissioner told The New York Times and a local newspaper that she suspected the decline had less to do with the spread of the virus than the behavior of those who’d been infected. She said she’d heard that fewer people with symptoms or exposure were agreeing to be tested at all.
Nuzzo points out that the ratio is especially distorted in communities where people may not have access to testing or where people may not want to be identified as being Covid-positive because they are afraid of losing their jobs. “And these are communities that happen to be hard-hit by the virus,” she says.
It’s also subject to reporting inconsistencies. Nuzzo and her colleagues at Johns Hopkins are wrestling with the fact that different US states calculate their test positivity ratios in different ways: Some report the number of positive tests divided by the number of tests done; others report the number of positive tests or positive people divided by the number of people who take any tests at all.
There are other, better options. While California’s Department of Public Health assigns every county a color-coded risk status based on the local test positivity and rate of new infections, a recent batch of stay-at-home orders kicked in only when the available capacity at their intensive care units dipped below 15 percent. Many other locations are following suit and setting policy according to the extent to which hospitals are overstretched. That measure tells planners something they need to know—that preventive measures are failing.
Once you’re counting hospitalizations, though, or the number of empty ICU beds, the surge of cases is already well under way. A better indicator of an outbreak may be increases in the number of positive tests, which works as long as sufficient testing is available and accessible. Or else there’s New York City’s replacement for test positivity: Frequent random screening within the most relevant population. And in some places, public health authorities are keeping an eye on viral counts in sewage to tell them when the virus might be on an upswing.
In the end, the best one can say about test positivity is that it gives a sense of how well a community is doing with testing, and that provides some kind of benchmark, however flawed. “It allows you as a policymaker to say, right or wrong, look, we are here for you. We are keeping an eye on this. We have this metric,” says Mina. Now, at last, there’s better information, and the era of test positivity as a stand-in for a real count of Covid-19 cases may soon be over.
📩 Want the latest on tech, science, and more? Sign up for our newsletters!
The “healthy building” surge will outlast the pandemic
The perfects strategy to fight Covid-19 is … everything?
I tested positive. What does that really mean?
Who will we be when this is all over?
Get WIRED: A letter to my pandemic baby
Read all of our coronavirus coverage here
