Why Paul Ryan Has the Toughest Job in Washington

By any rights, Paul Ryan should be at the peak of his game. He’s the Speaker of the House at a time when Republicans control both chambers of Congress and the White House, and this fall he has the opportunity to spearhead the first reform of the tax code in three decades, an issue he’s long promoted.

But things aren’t quite working out that way.

In recent weeks, Ryan has faced criticism from House conservatives even as he was undercut by President Trump on a deal with Democrats to extend the debt ceiling—on the same day he called the idea “ridiculous.”

Now, there are reports that some conservatives are considering a challenge to his speakership. The Washington Post reported last week that House conservatives were thinking of trying to depose Ryan, and the Wall Street Journal editorial board even wrote that potential challenger Rep. Mark Meadows should “man up” and run now rather than hurt Republican priorities from the sidelines.

As with similar rumors during the previous Republican speaker, John Boehner, this talk won’t likely amount to much, and the key players have said they have no interest in it.

“There’s no plan to change leadership,” said Meadows, the North Carolina congressman who chairs the far-right Freedom Caucus. “But there’s a concerted effort try to make sure we get things done. I’ll be very clear about that.”

He added: “If I was involved in some kind of plan to change leadership, you guys wouldn’t be reading about it or hearing about it.”

But the fact the rumors have even been floated is not a good sign.

“There has been a lot of frustration with leadership throughout the conference,” said one source with the Freedom Caucus. “There’s a sense that things might need to change if things don’t get done by the end of the year.”

Still, a major problem for Republicans unhappy with Ryan? There’s no clear alternative who could get a majority of the GOP. “Who’s gonna take his place?” Steve Bell, a senior advisor at the Bipartisan Policy Center, says. “Mark Meadows? Jim Jordan? Not likely.”

The most recent trouble began last week when a reporter asked Ryan about reports that Democratic leadership wanted to bundle relief funding for Hurricane Harvey victims with a measure to raise the federal debt ceiling for another three months.

“I hope they don’t mean that,” Ryan responded. “We’ve got all this devastation in Texas and another unprecedented hurricane about to hit Florida, and they want to hit politics with the debt ceiling? I think that’s ridiculous and disgraceful.”

An hour later came the announcement that shook the Hill: at a White House meeting with congressional leadership Trump had cut a deal with Senate Democratic Leader Chuck Schumer and House Democratic Leader Nancy Pelosi, agreeing to support a bill that combined Harvey relief with a debt ceiling increase and a continuing resolution to keep the government’s lights on until December—a variation of the very “ridiculous and disgraceful” plan that Ryan and other GOP lawmakers had denounced.

With Ryan left with egg on his face, conservatives were fuming. “It’s very frustrating,” Mark Walker, the North Carolina congressman who chairs the conservative Republican Study Committee caucus, told reporters. “Our leadership could do more to push back sometimes, to stay consistent with what we promised the American people when arriving in Congress.”

The debt ceiling deal was a blow to Republicans, who generally see raising the borrowing limit without corresponding spending cuts as fiscally irresponsible. And that Trump ignored his party’s concerns and publicly aligned himself with Pelosi and Schumer—two GOP bogeymen—added insult to injury, particularly when the Republicans are still smarting from its failure to repeal the Affordable Care Act.

The White House’s tensions with Ryan and Senate Majority Leader Mitch McConnell are old news at this point. On Aug. 10, after the final, flimsy attempt to dismantle Obamacare floundered, Trump tweeted on Aug. 10 that McConnell “couldn’t get it done.”

“What’s most remarkable is that the GOP congressional leadership and Trump managed to pretend to get along for as long as they did,” Philip A. Wallach, a senior fellow in governance studies at the Brookings Institution, tells TIME. “Their personal poor relations were waiting to blow up. Fundamentally the GOP leadership has a strong ideological commitment to smaller government, and Trump simply doesn’t have that.”

But the loudest rumblings about the future of Ryan’s leadership have been traced to the more conservative factions of the House—namely the right-wing Freedom Caucus and the Republican Study Committee. It was Meadows, the Freedom Caucus chair, who first sought to dethrone Speaker of the House John Boehner in 2015. Members of these groups share Ryan’s aversion to big government, but find themselves answering to constituents who disdain the Washington establishment.

“They do have strong ideological aversions to Trump, but a lot of them come from districts where Trump is phenomenally popular,” Wallach says. “It creates a real dilemma for them. They’ve tried to be very pro-Trump while blaming the congressional leadership for not standing up for their small government values.”

At a meeting of House Republicans on Friday morning, conservatives aired their frustration. “They were unhappy that they’d wanted a longer-term [debt ceiling] deal and the president undercut it,” one person in the room says.

For now, Ryan will remain the most convenient scapegoat. His next stress-test will be tax reform. Recent legislative history and the tensions between lawmakers suggests that this will be a struggle, and that leadership will shoulder the blame if it fails.

“I don’t have any reason to think Paul Ryan is exceptionally gifted or bad at managing those tensions. But he has one of the hardest jobs in the country,” Wallach says. “They haven’t gotten much at all done. It’s a huge disappointment, and it doesn’t line up with the rhetoric.”