Pressure on Good Science During a Pandemic Is Leading to Confusing, and Conflicting Advice on COVID-19

Research is normally a plodding, tedious process. Scientists check and recheck their data; review and re-review their conclusions; then submit their hard work to a scientific journal for publication, where their peers put it through further scrutiny. But a viral pandemic doesn’t adhere to a cautious timeline.

As COVID-19 has raced across the globe, public-health experts, political officials, doctors and patients scrambled to find answers about the disease. Digital sites that posted manuscripts of scientific papers before peer review have flourished since January, and editors of prestigious medical journals have asked their peer reviewers to complete their analyses, traditionally done over weeks, in just days.

That pressure to publish is exposing the tension between the desire within the scientific community to only release information once it has been fully vetted–a process that takes weeks and months–and the urgent public need for actionable information in the midst of a devastating pandemic. It’s not simply an academic matter for those in the research community; increasingly, policymakers have turned to the scientific process to guide their decisions–not to mention the doctors who trust it for finding ways to treat their patients and save lives.

The result is a confusing and often contradictory set of messages from public-health experts. On June 8, a World Health Organization expert declared that COVID-19 spreading from people without any symptoms was “very rare,” then admitted a day later that the conclusion was a “misunderstanding” and based on only a few studies. The concern over asymptomatic spread is a major reason behind social-distancing practices and advice for people to wear face masks in public.

Ever changing public-health advice is the cost of quickly collecting, digesting and making sense of information about a completely new virus. For doctors treating patients with COVID-19, the pressures of managing a pandemic illness are exacerbated by the fact that their North Star for treatment decisions–peer-reviewed reports in medical journals–are also facing steep challenges in finding ways to publish reliable information with far less time than they are accustomed to having.

In early June, for example, scientists from prestigious academic research institutes retracted two papers related to COVID-19, one involving hydroxychloroquine and another investigating blood-pressure medications, which were both based on data supposedly from patients in hundreds of hospitals on six continents. The scientists decided to pull the papers after the data-collection company they used refused to provide the peer reviewers the full set of data. “Without a doubt in the rush to produce manuscripts for peer review, I am concerned that investigators may be under the same pressure to rush their studies as journals are to publish them,” says Dr. Howard Bauchner, editor in chief of JAMA.

In calmer times, prestigious journals such as JAMA vet every submission through a team of editors and peer-review experts over several months. During this pandemic, however, the volume of papers has surged, and “there is no way for a traditional peer-review process to keep up with that,” says Jonathan Eisen, professor of evolution and ecology at the University of California, Davis. Rather than wait, scientists are funneling dozens of COVID-19 papers daily to preprint servers, or online repositories for scientific manuscripts that have not yet been peer-reviewed. That’s raising concerns about the risks of publishing unvetted studies related to patient care. “If there are mistakes in those studies, there is no capacity or limited capacity to correct them,” says Bauchner.

But it may be time to modify the scientific process so it can be more agile at providing experts with critical and reliable information in a timely manner. Encouraging researchers to openly share data used in their studies, for example, would allow more scientists to quickly evaluate the validity of their results. Eisen, who serves on the advisory board of a preprint database, is also in favor of making the peer-review system more transparent, and boosting the numbers of reviewers by asking graduate students and other qualified people to quickly audit studies. “There is no doubt that [the pandemic] is making it easier for bad players to play badly, but it’s also easier for good players to play well,” he says.