Is the U.S. COVID-19 Vaccine Rollout Working? We’re Not Sure Yet

After four months and 171 million doses of COVID-19 vaccines administered across the U.S., more than a few of us are eager to know: are the shots working?

Thus far, available evidence can half-answer that question: The vaccines are working well for those who can get them. As soon as the rollout kicked off, a variety of researchers began conducting what might be considered an unofficial phase 4 clinical trial, monitoring early vaccine recipients among the general public. Several of their studies have found a significant reduction in both the rates of infection and severity of symptoms among vaccinated populations compared to similar demographics who are still awaiting a dose. On an individual level, the vaccines appear to be working as well in real life as they did in controlled trials.

But there’s another part of that question: will mass vaccination hasten the end of the pandemic? On first pass, this might seem to be happening. As of April 7, nearly 20% of Americans have received a completed regimen of either the two-dose Pfizer-BioNTech and Moderna vaccines, or the one-shot Janssen variety. Meanwhile, the daily number of deaths in the U.S. attributed to COVID-19, on a rolling seven-day average, is currently 611—a ghastly figure, but a tremendous drop from the all-time post-holiday peak of 3,428 on Jan. 11. The last time the figure was this low was on Jul. 4, 2020, just before deaths climbed again in the wake of a second wave over the summer, from which the country never fully recovered before the much more catastrophic third wave spurred by the holidays.

Unfortunately, the researchers I’ve consulted were unanimous in saying that it’s far too soon to attribute the decline in deaths to the vaccine rollout on even a tentative basis. In part, that’s because vaccination rates are still too low to show a connection. Meanwhile, changes in the death toll typically lag behind changes in cases by several weeks—and when it comes to measuring whether mass vaccination can slow or end the COVID-19 pandemic, it’s a decline in the death toll that matters most.

“Mortality is probably the first place to look because death is a definitive event, unlike cases, even though we all know that COVID deaths are undercounted,” says Dr. Philip Landrigan, director of the Program for Global Public Health and the Common Good at Boston College, who previously worked in the Immunization Program at the U.S. Centers for Disease Control and Prevention. The critical question is whether the U.S. COVID-19 mortality figures will continue declining as vaccination accelerates, or whether the current dip is merely a temporary nadir ahead of what may very well be a coming fourth wave, as improving weather and misguided complacency lures more of the population out of their proverbial bunkers.

Even the most capable, data-rich biostatisticians say many more Americans will need to be vaccinated before we can measure the impact of mass inoculation. As an article in the journal Nature recently put it, “teasing apart the population-level effects of vaccines on a drop in COVID-19 cases from the impacts of other public-health interventions, such as social distancing and lockdowns, will be tricky.” Landrigan and other experts I consulted—while armed with a sheaf of scatterplots I made comparing each state’s vaccination rate to various COVID-19 metrics, looking for at least a crude correlation—said the same thing.

However, my largely fruitless examination of the state-by-state figures did turn up a few points of interest. Nearly every state is currently reporting mortality figures resembling the interregnum last summer or last October, between peaks that began in June and again in the fall. The exceptions are mainly a handful of states that have recently added previously overlooked COVID-19 deaths to their totals, which often creates an artificial spike in the data.

There’s also significant variation in the percent of the population that is fully vaccinated across the union, from a low of 14.1% in Georgia to a high of 26.4% in New Mexico. If this disparity persists, it may offer some clue as to the vaccines’ population-wide efficacy—assuming the trajectory of the mortality rate across the states aligns with their vaccination rates several months from now. But that organic experiment is likely to offer only a hint at best, given that populations, public health policies and other variables differ tremendously from one state to the next.

What will be considerably more telling, I suspect, is which direction the mortality rate bends in the three-way tug-of-war between the vaccination rollout, the mutation of the virus, and people (sometimes prematurely) reengaging with the outside world. Untangling those strands is further complicated by the fact that any effort by governors and other leaders to reduce COVID-19 mortality is like steering a giant boat: The results of turning the wheel are not immediate, and sometimes you get stuck in the Suez.

In the interim, researchers will continue to study both smaller populations with much higher vaccination rates, like nursing homes, as well as countries like Israel, which was at least initially successful in a widespread vaccination rollout, though the pace has since tempered.

It’s very easy to forget that we’ve been here before, nearly a year ago. With spring in full bloom and Memorial Day tantalizingly close, states at this time last year were drawing up schemes for reopening beaches and restaurants to at least a fraction of the original capacity. The result was catastrophic, with cases and deaths ballooning through mid-July. The need to accelerate the vaccination program before the lure of summer intercedes has never been more exigent—nor has the imperative to remain cautious and patient, even when the end appears to be in sight. After more than a year of hermitage, everyone is restless. And there’s no vaccine for that.