Putin speaks in Moscow in March at an event marking the first anniversary of the takeover of Crimea.
Maxim Shipenkov—The New York Times/Redux
August 27, 2015 6:24 AM EDT

Gleb Pavlovsky arrived for work as usual that day in the spring of 2011, walking up to the clock tower of the Spassky Gate, which serves as the entrance to the Kremlin fortress. This had been his routine during the first two terms of Vladimir Putin’s presidency, when Pavlovsky had served as a top adviser on matters of domestic politics and propaganda. But on that April day, Pavlovsky discovered that his security pass would not open the gate.

“They just locked me out,” he recalled this spring at his personal office, a shamble of books and papers on the top floor of a crumbling apartment block in central Moscow. Pavlovsky was hardly alone–in the years since his dismissal, many others have been discarded from Putin’s staff in the same way, especially the more politically liberal members of the ruling class, the ones who wanted to stop Russia from tumbling backward into another Cold War with the West. For them the past few years have been a period of setbacks and humiliations–“a shriveling up,” is how one Kremlin consultant put it–while the hard-liners in Putin’s circle have seen their influence steadily expand.

Known in Russia as the siloviki, or “men of force,” this coterie of generals and KGB veterans has come to fully dominate political life in Russia in the year and a half since the war in Ukraine ruptured Moscow’s relations with the West. Their rise has contributed to what several current and former advisers to the Kremlin describe as an atmosphere of paranoia and aggression. Officials seen as sympathetic toward the West have been mostly sidelined and discredited, limiting the voices Putin hears on matters of national and global security. The result is a regime in Moscow that looks increasingly antagonistic to the West and appears prone to ill-considered and dangerous decisions. “Sometimes the old instincts kick in,” says one of Putin’s senior counselors, referring to the Cold War backgrounds of the officials who now dominate the Kremlin. “I’d say there is the danger of going backward.”

That’s bad for an increasingly isolated Russia, but it’s dangerous for the entire world. Against the backdrop of the war in Ukraine, where Russian-backed militants have taken control of large patches of territory, both Russian and Western forces have dramatically ramped up their military exercises in Eastern Europe. The outcome “has been a game of Russian-instigated dangerous brinkmanship which has resulted in many serious close military encounters between the forces of Russia and NATO,” said a report published on Aug. 12 by the European Leadership Network, a think tank that monitors security threats in the region.

Should a mistake happen, it is far from clear that cooler heads would prevail in the Kremlin–for the simple reason that there aren’t many of them left in Putin’s entourage. Sergei Naryshkin, a close Putin ally and speaker of Russia’s lower house of parliament, suggested in a newspaper article on Aug. 9 that the U.S. is trying to goad Russia into war. In a warning to President Barack Obama, he wrote that it “wouldn’t hurt the current and latest ‘war-time’ President of the USA to remember: if you sow the wind, you will reap the storm.” Nikolai Patrushev, the head of Russia’s Security Council and a 17-year veteran of the KGB, was even more direct in an interview published in late June. “They really want Russia to cease to exist as a nation,” he said of the U.S. “Because we have enormous wealth, and the Americans think we have no right to it and don’t deserve it.”

Patrushev did not respond to numerous written requests for an interview with TIME, and most members of the siloviki have not spoken to the foreign media for years, which makes it difficult to properly gauge the opaque inner workings of the Kremlin. Yet watching these changes from a distance, Pavlovsky, like many other more liberal ex–Kremlin members, finds it hard to recognize the place where he worked just four years ago. Back then the Kremlin’s staff had a far more diverse makeup–liberal economists, dowdy intellectuals, bureaucrats with Western bank accounts and children studying in Europe or the U.S. Taken together, their influence balanced the more bellicose men of the siloviki, whose persistent warnings of the American menace are now the only voices Putin hears. Says Pavlovsky: “We have a situation where the person who does not immediately raise his voice to a scream is seen as suspicious.”

By the spring of 2012, when Putin began his third term as President, Kremlinology–the esoteric discipline of studying Russian power politics–needed an overhaul. Among the first attempts to address the system’s opacity came in the fall of 2012, when a well-connected Moscow political expert named Evgeny Minchenko, who has consulted for Putin’s party, United Russia, created a diagram of the ruling class titled Politburo 2.0. His most recent one, published last fall, resembles a spider’s web with Putin at the center. Clustered around him are various oligarchs, generals, spymasters and technocrats, their influence denoted by their proximity to Putin. Over the past year, he says, “the main trend has been an undeniable spike in the influence of the siloviki.”

What unites most members of Putin’s Politburo are the personal bonds they developed with Putin years ago in their hometown of St. Petersburg. As he rose to power in Moscow–first to the leadership of the Federal Security Service (FSB), the KGB’s successor agency, in 1998, then to the post of Prime Minister in 1999, and finally to the presidency in 2000–Putin brought his friends along. “He maintained that healthy sense of being one of the guys,” said Anatoly Rakhlin, Putin’s childhood judo coach. “He didn’t take the Petersburg boys to work with him because of their pretty eyes,” Rakhlin told the Izvestia daily in 2007, “but because he trusts people who are tried and true.”

During Putin’s first term as President from 2000 to 2004, the Kremlin was still full of holdovers from the administration of President Boris Yeltsin, and most of them were devoted to free-market reforms of the economy and collaboration with the West. Chief among them was Prime Minister Mikhail Kasyanov, who held the keys to Russia’s budget. But over time, Putin’s team from St. Petersburg took over the levers of power in Moscow, Kasyanov says. “By the time I left [in 2004], they were really divvying things up,” he adds.

Most of the top jobs in the security services, the government and the powerful state corporations went to the members of Putin’s St. Petersburg circle, which came to form the core of what Minchenko calls the Politburo 2.0. The structure of this body differs drastically from its Soviet incarnation. Whereas the old Communist Party bosses met regularly to decide the affairs of the state together, Putin keeps his circle divided into clans and factions that seldom meet all at once. This helps prevent any groups from creating a coalition against him, and it also “makes Putin indispensable as the point of balance,” says Minchenko. “Without him the system doesn’t work, because everyone is connected through him personally.”

But there are major drawbacks. As the rival factions compete for Putin’s attention, they tend to exaggerate the threats that Russia faces. The intelligence services, for instance, might overstate the threat from foreign spies, while the oil and gas tycoons might play up the danger of competitors in the energy market. When Putin meets separately with each of these factions, “he hears from all sides that there are threats everywhere,” says the political consultant Kirill Petrov, who has worked with Minchenko in mapping the elites. “It’s not a healthy atmosphere.”

One of the figures in Minchenko’s diagram, the senior counselor to Putin who spoke on condition of anonymity, concedes that this informal system of relationships breeds paranoia. But the system’s bigger flaw is its total dependence on just one man. “It is power without institutions,” says the adviser. “It means we have no solid ground beneath us.” The state is Putin, and Putin is the state.

But if those closest to Putin are dedicated to their President, they’re also dedicated to the spoils that come with power. And that’s why Western sanctions imposed in response to the Crimean land grab have not only isolated the Russian economy but also personally targeted Putin’s close associates, banning them from traveling or doing business in the West.

The logic of that punishment was simple. Most members of the ruling class in Russia–liberal or not–send their children to study in the West. They keep their fortunes in Western banks. They ski in the Alps, sunbathe in Miami and go shopping in Milan. For many of Putin’s allies, it is not worth risking such privileges for the sake of any extraterritorial ambitions in Ukraine, says Kasyanov, the former Prime Minister. “Of course this makes people question their loyalties,” he says. “Their lifestyles are at stake.”

That would seem to be especially true for influential tycoons like Gennady Timchenko, a wealthy oil trader from Putin’s St. Petersburg circle, whose personal wealth fell from $11 billion to about $4 billion last year, according to the Bloomberg index of billionaires. (The extreme drop in oil prices–now less than half what they were a year ago–has also weighed on many Russian tycoons.) But like many of his fellow oligarchs, Timchenko has supported Putin’s policies despite the pain. “It’s naive to think these methods can scare us, make us retreat,” he said in an interview last summer with the state news agency Itar-Tass. “We’ll bear it all and find a way out of these sanctions.”

Far from peeling off Putin’s allies, the sanctions have allowed him to tighten his grip on power. For the past few years, Putin has urged elites to store their fortunes in Russia instead of stashing them in offshore bank accounts. Many of them were slow to comply before the sanctions put their Western assets at risk of being frozen. Now the fortunes of the elites are tied that much more closely to Russia–which means they’re tied to Putin.

So if Western leaders were hoping Putin’s allies would mount a palace coup, they will likely be disappointed. The culture of suspicion has only intensified amid the standoff with the West. While the influence of the siloviki has grown, so has the number of subgroups vying to be recognized as the most loyal, the most effective at fighting Putin’s enemies. The only way a new leader could emerge from among them is if Putin himself starts to groom a successor.

He has no need to do that anytime soon. The 62-year-old Putin is expected to run again when his six-year term in office ends in 2018. Among the “men of force” rumored to be possible successors are Sergei Ivanov, his long-serving chief of staff and fellow alumnus of the KGB, and Sergei Shoigu, the Minister of Defense, who has played the most visible role, apart from Putin, in Russia’s military interventions in Ukraine. But in a system where all institutions have been eclipsed by one personality, there is no way to know what happens when he’s gone.

Of course, not even Putin is immortal. But while the Communist Politburo would meet to elect a new leader when the incumbent passed away in Soviet times, “in the current system, Putin has no answer to the question of what happens if he has a heart attack,” says the President’s counselor. He knows that a struggle for control would then break out among the factions in the Kremlin, and in the process, “some of his friends could be torn into slivers of flesh,” the adviser says. “So I don’t think it’s out of cleverness that he’s made everybody afraid of his departure. It’s just that he doesn’t know how to do it any other way.”

This appears in the September 07, 2015 issue of TIME.

Contact us at letters@time.com.

Read More From TIME

Related Stories

EDIT POST