Here’s the One Simple Way To Fix Weather Forecasts

Just after midnight on Tuesday morning, Gary Szatkowski, the meteorologist in charge of the National Weather Service’s office in Mt. Holly, New Jersey, took to Twitter for some good old fashioned self-flagellation.

“My deepest apologies to many key decision makers and so many members of the general public,” he wrote to his 14,000 followers. “You made a lot of tough decisions expecting us to get it right, and we didn’t. Once again, I’m sorry,” he added two minutes later.

Szatkowski, of course, was apologizing for his agency’s forecast for a blizzard of historic proportions that — at least in his part of the world and as far north and east as New York City — fell far short of expectations, leaving inches of snow where people were expecting feet.

New York City Mayor Bill de Blasio took a decidedly different approach to the miscalculation. “Would you rather be prepared or unprepared? Safe or unsafe?” he asked rhetorically at a post-snow press conference, where his agency heads proudly boasted of their heroic responses to an average winter storm.

So, who’s right? In an age of weather hype, with 24-hour news cycles and superlative-laden click-bait headlines, is it have better to under promise or plan for the worst?

The answer, according to Susan Joslyn, an Associate Professor of Psychology at the University of Washington who studies decision making in applied settings, is complicated.

In a 2015 paper, co-authored with Jared Le Clerc, Joslyn examined how the “cry wolf effect” impacted weather-related decision making. According to her research, increasing false alarms (like over estimating snow totals or issuing unnecessary travel bans) has a negative impact on the quality of decisions down the road, but reducing false alarms doesn’t have a positive impact on the quality of future decisions. Adding an uncertainty estimate to the forecast, however, does.

“My personal feeling about this is it is appropriate to take precaution, even if there’s a small chance, especially if human life is at risk,” Joslyn said during a telephone interview.

“But what authorities ought to do for decision makers and the public is tell them ‘yes, I think there’s a good chance’, and then tell them the probability. What they tend to do is say ‘It’s going to happen or ‘It’s not going to happen.’ Our research indicates that giving the full story has the best results,” she said.

Mike Smith, a meteorologist from AccuWeather Enterprise Solutions and a weather historian, thinks that Joslyn’s work is fine in theory but sees some real challenges to probabilistic forecasts in practicality.

“In this storm, which point do you pick to represent New York City? Is it the west side of Manhattan, the east side of Queens? There’s an 8-inch difference in total accumulation between the two,” Smith said in a telephone interview, adding, “And when you’re on TV, you can’t go on and on about how Staten Island has this likelihood of that amount and Brooklyn has that likelihood of this amount.”

“Perhaps as weather science advances,” he added, “and as the web becomes more ubiquitous, maybe. But with toady’s dissemination technology it’s very difficult to give probabilistic snow forecasts to everyone.”

What’s more, Smith thinks the forecast was just fine.

“The average snowfall in New York was 11”. Did they have had to load the trucks with salt? Yes. Did school have to be called? Yes. Would workers have been told to stay home or telecommute? Yes. Would flights have been cancelled? Yes. Once you reach that threshold it really doesn’t matter how much more snow falls.”

“Was it an excellent forecast for points south and west of New York City? Of course not, but it was useful. For everyone north and east, I’d say that it was lifesaving. To me, that makes it a good day for meteorology.”