Many Metrics to Measure COVID-19 -- Which Are Best?

Editor's note: Find the latest COVID-19 news and guidance in Medscape's Coronavirus Resource Center.

You've seen the debates, on television or on social media, or even in your own conversations.

They go something like this: "We should reopen (schools, cities, states, countries) because the number of daily cases is down!" one person says. "No, no, you have to look at the death rate! That's a lagging indicator and is still going up!" says another person. "And our hospitalization rate is still way too high!" a third person chimes in.

In this pandemic, there are many different metrics used to measure the situation. Each has its own usefulness and its own limits.

The metrics used to track the coronavirus pandemic typically include daily cases, hospitalizations, and deaths. Analyzing these metrics separately can show how much community spread there is or whether hospital capacity is being reached.

"Metrics serve different purposes — it depends on the purpose for using the data," says Amesh Adalja, MD, a senior scholar at the Johns Hopkins University Center for Health Security in Baltimore.

The University of Washington's Institute for Health Metrics and Evaluation (IHME) makes forecasts based on what is known about a disease and how people's actions may affect that.

The IHME's latest COVID-19 forecasts say the U.S. will reach nearly 317,000 deaths by Dec. 1, at the current rate of mask-wearing, which dropped to slightly below 50% nationally last week. But increasing mask wearing in public to 95% could save more than 67,000 lives, says Ali Mokdad, PhD, a professor of health metrics sciences at the IHME.

"Forecasts are not static but can change depending on public behavior," says Mokdad, who's also chief strategy officer for population health at the University of Washington. When people learn that new cases are rising, they start wearing masks and using social distancing again; and when they realize new cases are declining, they tend to drop their guard, he says.

New cases surged when governors lifted lockdowns in several states in the Southeast and Southwest in the spring. At least 34 states have now mandated statewide mask wearing.

To create the forecast, the IHME uses real-time infection data from Johns Hopkins University's Coronavirus Resource Center to model disease transmission and project how many Americans will die. The researchers then estimate how many Americans are wearing masks or using social distancing, which can change the final model.

Measuring COVID-19 Transmission

Researchers estimate the rate of infection in a population based on the "R0," or reproduction number. R0 is the average number of people who will catch the disease from a single infected person, in a population that's never seen the disease before. So, if R0 is 3, that means one case will create an average of three new cases. When that transmission rate of infection occurs at a specific time, it's called an "effective R," or "Rt."

When the R0 is less than 1, that means the epidemic is under control; and when it's higher than 1, it is still spreading.

When the IHME analyzed the combined data on cases, hospitalizations, and deaths for the week ending Aug. 27, it found transmission increasing in a cluster of states in the Upper Mississippi Basin, including Iowa, Indiana, Missouri, Kentucky, and Tennessee. The "effective R is also over 1 in Oklahoma. In all other states the effective R is less than 1."

For the CDC, COVID-19 cases come from positive tests results. Websites that track COVID-19 often report these as confirmed cases.

But just looking at raw case numbers won't tell you how much of the population is infected, says Adalja, the Johns Hopkins senior scholar. "You have to adjust or control for that population size by using one case per 100,000 people. This also allows valid comparisons with other states with different population sizes."

The positivity rate indicates how hard or easy it is to find a case, which reflects both the spread of COVID-19 and how widespread testing is, says Adalja.

"If the rate of positive tests is 20%, you don't have to look hard to find a case, versus 1%, which means you have to do a lot of tests to get one positive one."

The more COVID-19 spreads, the higher the positivity rate.

But "context is important," Adalja says. "A 60% positivity rate may mean testing is only being done in a nursing home during an outbreak or a hospital where the most obvious cases are and not the general population where cases may be milder."

Maryland's COVID-19 dashboard reports the daily positivity percentage, which is the percentage of positive tests and total testing volume since March.

"When you're looking at testing, you want to know how many tests were done historically with the ability to compare back and know whether the number has gone up or down or is stable and the percentage that comes back positive," says Adalja.

Maryland and Pennsylvania report a 7-day rolling average of the daily positivity percentages. "The 7-day average rate smooths out fluctuations during the week and is a better indicator of a trend than daily numbers," he says.

The testing numbers often fluctuate, depending on where testing is done and when the labs report test results. A sudden spike in testing numbers may reflect a large number of tests done in a group setting such as a nursing home or prison on a single day. Laboratories and hospitals report test results on weekdays, so it's common to see those numbers decline on weekends.

Hospital Capacity

A key goal during the coronavirus epidemic has been to "flatten the curve" to maintain local hospital capacity. After expected COVID-19 surges, many hospitals limited surgeries and admissions to preserve their resources, including hospital beds, ventilators, and health care personnel.

"You want to protect your hospital capacity. If that reaches 80%, you may have to stop admitting patients; otherwise, the hospital may be overwhelmed," says Mokdad, the IHME professor.

To plan for surges and increase capacity, administrators should know the number of people who tested positive and were admitted to the hospital with symptoms of COVID-19, he says.

Knowing the number of beds available also helps hospitals plan for surges. Pennsylvania's COVID-19 dashboard has a hospital preparedness page that lists the number of hospitalized COVID-19 patients and the number and percentage of available beds by unit, including intensive care, medical/surgical, and airborne isolation.

Pennsylvania's dashboard also reports the number of ventilators COVID-19 patients and non-COVID-19 patients use daily.

States like Illinois list the recovery rate from COVID-19 on their dashboards. In Illinois, the recovery rate of 95% is calculated as the recovered cases divided by recovered cases plus confirmed deaths. "This [metric] is important because it indicates the quality of medical care and the severity of disease," says Mokdad.

Deaths

The ultimate goal of any epidemic response is to save lives, so monitoring death counts due to COVID-19 is important, especially when testing is limited, according to the Johns Hopkins University Coronavirus Resource Center, which developed management metrics for cities.

For example, states count "probable" or "presumptive" COVID-19 deaths when cases are not confirmed with a positive test but are based on symptoms and medical history. For example, New York added 3,700 presumptive deaths in one day in April when testing was more limited, says Mokdad.

The IHME says daily deaths are "the best indicator of the progression of the pandemic, although there is generally a 17- to 21-day lag between infection and deaths."

Sources:

Ali Mokdad, professor of health metrics sciences, Institute for Health Metrics and Evaluation; chief strategy officer for population health, University of Washington, Seattle.

Amesh Adalja, MD, senior scholar, Johns Hopkins University Center for Health Security, Baltimore.

NPR: “Joe Biden: For The Next 3 Months, All Americans Should Wear A Mask When Outside.”

Institute for Health Metrics and Evaluation: “COVID-19 estimation updates.”

Institute for Health Metrics and Evaluation COVID-19 Projections.

AARP: “State-by-State Guide to Face Mask Requirements.”

The Atlantic: “The Deceptively Simple Number Sparking Coronavirus Fears.”

The New York Times: “N.Y.C. Death Toll Soars Past 10,000 in Revised Virus Count.”

Johns Hopkins University Coronavirus Resource Center: “Management metrics for cities in the COVID–19 crisis.”

Maryland Department of Health COVID-19 Dashboard.

Pennsylvania COVID-19 Dashboard.

Illinois Department of Public Health: “COVID-19 Statistics.”