Moderna’s COVID-19 Vaccine Is 94.5% Effective. Here’s What That Really Means

It’s wasn’t a typical Sunday morning for Dr. Stephen Hoge, president of the biotech company Moderna, and Dr. Anthony Fauci, director of the National Institute of Allergy and Infectious Diseases. They were at their respective homes in Massachusetts and Washington, D.C., waiting to to be let into a Zoom call to hear the results of the very first COVID-19 vaccine that was tested in people.

The hosts were members of the independent Data Safety Monitoring Board (DSMB) that is reviewing data involving all the COVID-19 vaccine candidates supported by the U.S. government’s Operation Warp Speed program, and Hoge and Fauci were expecting the call to be about the first results from a vaccine Moderna and NIAID had developed together. With the pandemic continuing to infect record numbers of people around the world, a lot hinged on what the DSMB would say.

The DSMB includes leading experts in the fields of infectious disease and vaccinology, as well as biostatisticians and ethicists, and is reviewing the first batches of study data for the first hints of efficacy. None of the people volunteering for the vaccine studies, nor the doctors running them, nor the companies sponsoring them, know what is being injected into the arms of the participants—half get an experimental COVID-19 vaccine, and half get a placebo placebo—so nobody knows if the vaccines being tested are working or not.

Except the members of the DSMB. They are the only ones that can “unblind” the results, at pre-specified times, to see if there is any hint of effectiveness. A week before Hoge and Fauci sat waiting for their Zoom call, Pfizer broke the ice when, on Nov. 9, it became the first to report on its preliminary vaccine data. The company’s safety board reviewed Pfizer’s late-stage, phase 3 data and determined that the company’s vaccine, which uses similar technology to Moderna’s, was more than 90% effective in preventing serious COVID-19 disease.

Now it was Moderna’s turn.

“It was the most nerve-wracking hour of a nerve wracking week of a nerve wracking year,” says Hoge, of the moments before the call. “You run through all of the scenarios in your head, and not all of them are good. In fact, most of them are bad.” Finally, the DSMB let Hoge and Fauci onto the Zoom call. “The moment they said there were 95 cases [of COVID-19], and 90 of those cases were in the placebo group, I almost didn’t hear the rest,” Hoge says. “I think I just broke out into a big, stupid-looking grin.”

Hoge couldn’t keep the smile off his face the next day either, as he discussed in a virtual call with TIME the implications of the results and the promise of the technology behind the vaccine. In a press release reporting on the DSMB’s review, the company said among the more than 30,000 people enrolled in the study, there were 95 cases of COVID-19 analyzed so far, and 90 of them occurred in the placebo group. That amounts to 94.5% effectiveness in preventing COVID-19 disease.

“This is definitely fantastic news,” says Dr. Evan Anderson, principal investigator of the vaccine’s study at Emory University. Since he was blinded to the results, Anderson says he didn’t even hear about them until he listened to the news on Monday morning. “This is really as good a response as we could possibly hope for. This puts it in the range of other highly effective vaccines, like measles.”

Carol Kelly, one of the first people to volunteer for an early safety study of the vaccine conducted at Emory, where she works as assistant director of nutrition services, was equally buoyed by the news. “I’m thrilled, and I feel validated; even though I’m just a participant, I’m amazed at the researchers and what they have done. I feel like it’s such great news, and something to get excited about.”

It wasn’t a given that the vaccine would prove so effective against the new coronavirus. Viral respiratory diseases are notoriously difficult to control with vaccines, because the viruses that drive them are so adept at mutating to resist any attempts to corral their spread. “For respiratory viruses, we have no history of anything near this efficacy,” says Dr. Lisa Jackson, who oversaw the first, Phase 1 study of the Moderna vaccine and injected the first participant in that trial. “If you think of influenza vaccines, they are around 50% efficacious.”

Fauci, too, was shocked in the best way when he heard the magnitude of the vaccine’s effectiveness. “I would have settled for a 70% or 75% effective vaccine; I think that would be terrific,” he says. “I had said that 90% or 95% would be aspirational but quite frankly I didn’t think we were going to have a vaccine with that high a degree of efficacy. I was very, very, very pleasantly surprised and admit I did not predict it would be that good.”

It wasn’t even clear whether this particular vaccine technology, which is new and has not resulted in an approved vaccine yet, would work at all. Both Pfizer and Moderna rely on mRNA, the genetic code that SARS-CoV-2, the virus that causes COVID-19, uses to make the critical proteins it needs to find, bind and infect cells. Once SARS-CoV-2’s genetic code was released in January by Chinese scientists, “the very next day we had a meeting with our group and said ‘let’s go for it,’” says Fauci. “We took a chance on the technology because of the advantages of flexibility of this platform, and the ability to rapidly go from [having the genetic] sequence right into the vaccine, which we literally did in days.”

In just over a month, the NIAID scientists had narrowed down the specific parts of the virus’s genome that they thought coded for the strongest and most obvious red flag—the spike proteins that stud the surface of SARS-CoV-2—for the human immune system. In another two months, they injected their first human volunteer in a safety trial. Both stages were reached in record times. “The word game-changer gets overused, but I do think we will forever think differently about how we respond to pandemics and emerging infectious disease threats,” says Hoge. “It’s hard not to notice that in the last 10 months, mRNA has gone from an idea with great promise to something that different companies are now using to create a new vaccine.”

Pfizer reported its mRNA vaccine was more than 90% effective in protecting against COVID-19 disease, but it’s important to understand what effectiveness means in their and Moderna’s studies. Both measured their vaccines’ ability to prevent COVID-19 disease, not to stop infection from the virus. In Moderna’s study, there were strict criteria to determine if and when people were tested for COVID-19. People in the studies needed to have at least two symptoms of the disease—including fever, cough, headache, shortness of breath muscles aches—or needed to have pneumonia before they could get tested. But some people can be infected and never develop any symptoms, and in both the Pfizer and Moderna studies, these people would not have been tested.

That means that the study did not necessarily compare how many total infections there were among those getting the vaccine and those receiving a placebo. For now, the 94.5% efficacy refers to the fact that among those who tested positive for COVID-19, the vaccine protected that percentage of people from getting disease v preventing them from getting infected with the virus in the first place. Hoge says that people with asymptomatic disease may be captured as people are followed for longer periods of time, and they are tested for antibodies to the virus, which would indicate if they had been infected with SARS-CoV-2, even if they never showed symptoms.

Before it can file a request to the Food and Drug Administration (FDA) for emergency authorization, Moderna still needs to wait for some additional data to be collected on all of the volunteers. The FDA requires that all vaccine makers submit two-month follow up data on study participants, to ensure that there are no adverse side effects from the shots. Hoge says the last participants will reach that two month milestone in a few days.

Now, with the positive results from both Pfizer and Moderna, attention will start shifting toward the next challenge in addressing the pandemic: making and distributing the vaccines. Moderna’s vaccine needs to be stored at -20° C, which many freezers at health clinics and hospitals can accommodate. Pfizer’s vaccine, on the other hand, needs to be kept at -75° C, so the company plans to ship its vials in thermal containers that can maintain that temperature for up to 15 days with dry ice.

Hoge says Moderna is on track to produce the 20 million doses it has promised to the U.S. government by the end of the year, and will continue manufacturing throughout 2021 to produce as many as 1 billion doses to distribute globally. State health departments are currently working with the Centers for Disease Control, which will oversee storage and shipping of vaccines, to determine when doses will be available, and in what amounts. Based on CDC and National Academy of Sciences recommendations, the first doses of vaccines will go to those at highest risk of COVID-19, like health care workers, first responders, the elderly, and people with existing chronic conditions that make coronavirus infection more dangerous.

Even after the first doses show up at hospitals, clinics, pharmacies and doctors’ offices, convincing people to get vaccinated will pose another challenge. People are especially wary about the mRNA technology that the Pfizer and Moderna shots use since it’s new and has been developed and tested so quickly; some aren’t convinced that the shot has been adequately tested. Trust in public health and government leaders also plays a role, as pandemic responses in places like U.S. have become politicized. In a recent survey involving people from 19 countries that represent around half of the world’s population, researchers found that the more people trusted their government, the more likely they were to be willing to get vaccinated with a COVID-19 shot.

“A vaccine in a vial is 0% effective,” Dr. Nadine Rouphael, interim director of the Hope Clinic of Emory Vaccine Center, said during a briefing. “We really need to make sure that the American public, and the global population at large, will be willing to take a vaccine.”

And even after they do, we need everyone to continue following public health measures to contain the spread of COVID-19. Because the vaccines haven’t yet shown that they prevent infection in the first place, “we can’t really abandon public health measures until the level of infection in a community is so low that the risk of getting infection is miniscule,” says Fauci. “The big message is that we have an additional tool [against COVID-19 in the vaccine] but we don’t have a tool to replace everything just yet.”