No COVID-19 Models Are Perfect, But Some Are Useful

The global coronavirus pandemic has brought renewed interest and focus on scientific models as we try to get a handle on what the future will bring, how many people will fall sick and die, what the economic impacts will be, and what actions politicians should take. But confusion abounds about what these “models” say and how to reconcile their often seemingly conflicting visions of the future. Recent political attacks on these models reflect a lack of understanding about what models are, how they work, and their usefulness and limitations. Conservative Fox News commentator Laura Ingraham attacked models on her show. Senator John Cornyn, a Republican from Texas tweeted “After #COVID-19 crisis passes, could we have a good faith discussion about the uses and abuses of ‘modeling’ to predict the future?”

Models are all around us. Without knowing it, we all use models all the time to try to understand outcomes of complex situations. The decisions you make on how to spend your monthly paycheck or save for retirement are financial models. The car you drive and the toaster in your kitchen were both designed with engineering models. Advertisers make models of consumer behavior, preference, and consumption of media when they design and buy ads. These models depend on science, but also on human behavior and actions that are far less predictable.

Even a recipe for bran muffins is a model—and a good example of the kind of models we use every day without thinking about it. Cooks combine centuries of knowledge about the chemical behavior of different ingredients with their personal experiences to create a model—a recipe—for what they hope is a delicious bran muffin. But whether the “bran muffin model” actually produces a good muffin or a burned hockey puck depends not just on the recipe but on factors completely out of the control of the recipe designer. When you set your oven for 400°F, does it heat to only 350°? Will you mistake a teaspoon of salt for a tablespoon of baking soda? Will you fall asleep and burn the muffins? Will the recipe maker’s tastes match your own?

Because of these uncertainties and unknowns, scientists who work with models try not to call the outcomes “predictions”—rather we call them “projections” or “scenarios.” A prediction implies more accuracy and certainty than many models provide. For all these reasons, scientists often repeat the classic aphorism “All models are wrong, but some are useful,” by which we mean models are only as good as our understanding of the scientific knowledge that goes into them. But useful models help us understand how science and human choices interact, providing valuable insight for policymakers.

Think about human-caused climate change: projections of climate change are based on some of the most complex models run on some of the fastest computers in the world. The climate is the most complicated biogeophysical system on the planet, affected by factors as varied as the output of the sun and tilt of the earth; the composition of the atmosphere; the behavior of winds, clouds, and ocean currents; interactions between the oceans, land, and atmosphere; and the behavior of plants and animals. Despite this complexity, global climate models are remarkably accurate, able to reproduce in tremendous detail the behavior of past and present climates. But their ability to produce accurate projections depends on not just getting the science right, but on assumptions about the future behavior of politicians and individuals; the role new technologies could play in altering greenhouse gas emissions; and the implications of investment, energy policies, and land-use decisions. We know that humans are already influencing and altering the climate, but the wide range of future climate projections largely depends on social and political uncertainties, not scientific ones.

Scientists are now building models to try to understand and project the path of the pandemic. How many people will get sick or die? What will be the consequences for society and the economy? How will different public policies, medical choices, and personal decisions influence the outcomes?

The models will continue to improve based on our developing scientific understanding of the infectiousness of the virus, the survival rate of the virus in different environments, the reaction of individuals to infection, the role of pre-existing conditions, the effectiveness of medical interventions, and much more. Less predictable inputs are those that attempt to understand the behavior of individuals: questions like will we stay home or go out; will we wear masks and maintain social distancing if we do go out;, or will we ignore medical advice and accelerate the “reopening” of the economy.

The takeaway here is that we shouldn’t dismiss one model just because it offers a widely different picture of the future than another. Don’t look at the divergent projections from these models and conclude the models are bad. These models are critical to help us explore the most effective actions to take to minimize what we care about: deaths, illnesses, and a damaged economy. The ultimate outcomes of the pandemic will depend on how we take the projections from the models and change our own behavior to avoid the bad futures we can see but desperately hope to avoid.