George Washington Made History by Deciding to Leave the Presidency. But That Didn’t Mean He Left Politics

The time of year around Washington’s Birthday — the federal holiday better known as Presidents’ Day — is a popular time for new biographies take a fresh look at America’s first presidency. But Jonathan Horn’s Washington’s End: The Final Years and Forgotten Struggle zooms in on what came after: the less talked-about last two years of Washington’s life, from the time he left office on March 4, 1797, to his death on Dec. 14, 1799.

Horn spoke to TIME about how, far from removing himself from politics after his two terms were up, Washington was in the thick of it — and torn about his legacy, especially on slavery.

TIME: What was the most surprising thing you found in your research?

HORN: We have this idea that George Washington finished his presidency in March 1797, rode away from Philadelphia [the nation’s capital back then] and went back to [his plantation] Mount Vernon. This is where the myth begins. We think we lived out his days peacefully as a farmer, but that’s not at all what happened. That’s what he wanted to happen. A little bit more than a year later he would find himself drawn back to the public stage.

Why did the myth about his peaceful retirement come about?

I think a couple of reasons. Washington did so much earlier in life — the American Revolution, the Constitutional Convention, two terms as President — that biographers are running out of pages and space, so essentially these last few years get left out. It’s also a complicated understanding of Washington. We want to believe he left power as the American Cincinnatus, that he gave it up and died, completely at peace. It’s really difficult to leave power. Power is a fragile thing. When Washington did it, there really were no precedents for how it would go.

I was struck by how so much of Washington’s post-presidency deviates from the way we think former Presidents should act today. We have this idea that former Presidents should become less partisan when they leave office. And without doubt he became more partisan when he left office. We also have this idea that former presidents shouldn’t meddle in the affairs of successors, but George Washington majorly meddled in the affairs of his successor and part of that was because he was asked to retake the role of Commander in Chief of the Army during a crisis.

What was the crisis?

The French were essentially the mightiest military power in the world. The French had been unhappy with the Jay Treaty that George Washington had signed and ratified with the British, and their ships and privateers were preying upon American ships. When America sent envoys to France they were refused. In the XYZ affair, basically the French asked for a bribe in return for receiving America’s ministers before they began negotiating. At that point, the U.S. began preparing for war. It wouldn’t be a formal war, we’d call it a “quasi war.” George Washington was asked to be Commander in Chief of the Army with the idea there could be an invasion by the French.

We tend to talk about the Founding Fathers warning about political parties, but it sounds as if they were deeply partisan.

Absolutely. I think they all stumbled into political parties. They didn’t want them to happen and Washington didn’t want them to happen most of all. He held out for a very long time, but in the last year of his life he’s actually calling himself a Federalist. He’s playing a much more active role in recruiting candidates for congressional elections. He would never have done that as President.

Get your history fix in one place: sign up for the weekly TIME History newsletter

Are there parallels to modern politics in the last years of Washington’s life?

Washington and his fellow Federalists strongly believed that the Republican Party [which is what the Democratic-Republican Party was called at the time, and unrelated to the modern Republican Party] was encouraging France in some sense. The French actually interfered in the election of 1796 on behalf of the Republican party, on behalf of Jefferson, in the hopes of getting him elected president. So those warnings against letting a foreign power interfere in domestic politics in Washington’s farewell address were based on real events that were happening. In some sense the first contested election in U.S. history was an election where there was foreign interference, so there are real parallels to Russia today.

There was also, during that period, a real explosion of partisan newspapers, and there was a feeling that new forms of media were going to tear the country apart. Political parties really started to emerge. There was a feeling that the country couldn’t stay united, similar to the way Americans feel today with divided media and divided political parties.

Was there anything Washington messed up, failed at or miscalculated in his last years?

He supports the Alien and Sedition acts, which sent journalists to prison. I think that’s left out of a lot of accounts of his life. He considered a lot of those journalists to be dangerous figures, and he was bitter about the way their colleagues attacked him personally.

In the last years, working on his will, he is still grappling with the issue of slavery, and the contradictions of slavery. At Mount Vernon, he can’t find a way to set free the slaves who came to Mount Vernon because of his marriage to Martha Custis, the so-called “dower slaves.” Many of his own slaves have married the dower slaves, and those marriages will be broken up. But even as he’s grappling with those issues and realizes those issues are painful, he’s looking for new ways to keep his slaves busy at Mount Vernon. He’s trying to pursue runaway slaves. You have many contradictions. He’s going to try to correct his stance on slavery with his will, but at the same time he’s still pursuing runaway slaves.

What would Washington think of Presidents’ Day?

Washington’s birthday was celebrated during his presidency and during the end of his life, and it was a bigger deal then than it is now. There would be cannon fire in the morning. During his presidency, people would line up outside his house and pay their respects to him, and at night there would be balls.

Washington would be surprised we call it Presidents Day and that he’d share his birthday with men he wasn’t getting along with. When he dies, he’s no longer on speaking terms with Thomas Jefferson, James Madison or James Monroe. So there’s some irony that Washington’s birthday is celebrated as Presidents Day.