Our Brains Make Us Way Too Optimistic About Meeting Deadlines. Here’s How to Work Around That

Not long ago, I visited a peculiar farm in southwestern Oregon. Year after year, Hastings Inc. produces a single crop: the Easter lily. Every Easter weekend, hundreds of thousands of lilies from this farm appear in supermarkets, big-box stores, and garden centers throughout North America. Each one has to look the same—a single stem, a dark green nest of leaves, and five or more flared white trumpet blossoms—and each one has to bloom at exactly the same time. They can’t miss their target by even a few days. As one of the farmers told me, “The day after Easter, an Easter lily is worthless.”

Why was this lily farm so good at meeting its deadline while the rest of us struggle endlessly? I found a clue on the wall of the main office at Hastings, near a window that overlooked the Pacific Ocean: a laminated calendar that showed the date of Easter for every year from 1996 to 2045. That calendar, I discovered, held the secret for how the farmers avoided an all-too-human problem known as the planning fallacy.

The term was coined in 1977, when Amos Tversky and Daniel Kahneman wrote a paper about predictions for DARPA, the Defense Advanced Research Projects Agency. Kahneman, who won a Nobel Prize in economics for the work he did with Tversky, later said the paper was inspired in part by an experience he’d had writing a textbook with a group of academics. At the beginning of that project, he had asked the participants to estimate how long it would take. The average guess was two years. It took nine.

Most of us are optimists, which might make us better company at the dinner table, but it means we are lousy at predicting the future. We underestimate the amount of time a project will require. If it’s a project that has a budget, we underestimate the expense as well. The most famous example of this failing probably the Sydney Opera House, which was commissioned in 1957 with an expected completion date of 1963 and a budget of $7 million Australian dollars. The building wasn’t finished until 1973, and only after the most ambitious versions of the plan had been scaled back, for a final cost of $102 million.

The planning fallacy is the tendency to seize upon the most optimistic timetable for completing a project and ignore inconvenient information that might make you revise that prediction. According to Roger Buehler, a professor of psychology at Wilfrid Laurier University, people are pretty stubborn about these conclusions even when presented with evidence of how they’ve been wrong in the past. Although people are aware that “most of their previous predictions were overly optimistic, they believe that their current forecasts are realistic.”

More from TIME

Buehler and some colleagues at the University of Waterloo in Ontario ran a test on their students to see how bad they were at estimating when they would finish their work. They asked thirty-seven seniors to make three predictions: the date they would submit their honors thesis “if everything went as well as it possibly could,” the date “if everything went as poorly as it possibly could,” and their best guess for what their actual submission date would be.

Fewer than 30 percent submitted their work by the date they thought was the best estimate of when they would be done. The optimistic predictions were even worse—they were off by an average of 28 days, and barely 10 percent of the students were done by that date. The most striking result, however, might be the one for the pessimistic scenario. Even when asked to predict what would happen “if everything went as poorly as it possibly could,” the students were still too optimistic. Fewer than half had finished by the worst-case-scenario date.

The problem with our predictions is that we treat each task like it’s a novel problem. We construct a story about how we will complete our work but ignore evidence from similar projects we or other people have done in the past. That was true with Kahneman’s textbook: One of the academics later admitted that earlier projects he’d worked on took a minimum of seven years. But when it was time to estimate how long this one would take, he guessed two years like everyone else.

It’s not all hopeless, though. There is a way to overcome, or at least mitigate, the planning fallacy. In a follow-up experiment, Buehler and his colleagues had a different group of students complete a one-hour computer tutorial at some point before a one- or two-week deadline. They also asked them to predict when they would finish the assignment, but here the researchers inserted a variable. Some of the students were prompted to think about past assignments they had completed that were similar to this one, and to apply that knowledge to their prediction. The control group was given no such instructions.

The results were remarkable: although the control group exhibited the same optimistic bias as the students in the first experiment, that bias almost disappeared among the students who were prompted to forge a connection between their past experiences and the current assignment. They predicted it would take them an average of seven days to complete the tutorial. The actual average: seven days.

In that office on the lily farm, there was a reason that a calendar dating back to 1996 was up on the wall. It was a cure for optimism. The farmers were bound by the Easter deadline—they couldn’t afford to let the planning fallacy have its way with them. So they did what those students did, only without the prodding of a group of professors. They took their past experience and used it to build a schedule, counting back from Easter. They knew down to the day how long the lilies had to be in greenhouses, how long in storage, and how long it took to get them out of the ground and into boxes. The calendar was there to remind them to think about how it was done in 1996, or 2006, or 2016.

When it comes time for you to build your own Sydney opera house, even if you don’t have a deadline as clear-cut as Easter Sunday, take a cue from those lily farmers. Ignore what you wish were true. Use the past to build your schedule. Keep your eye on the calendar—and then watch those flowers bloom.

This article is adapted from Cox’s new book, The Deadline Effect