Is the White House a ‘Dump’? It Used to Be

The White House is often seen as a symbol of American strength and splendor — even though President Trump reportedly called the executive mansion “a real dump,” according to a new report from Sports Illustrated. And yet there have been moments when the nation’s fortunes were unable to make it as grand as its residents might have liked.

At various points in its past, the White House has nearly crumbled around the Presidents who called it home. In this excerpt from the 2016 TIME special edition Inside the White House, available on Amazon, look back at how the White House went from ramshackle mansion to modern marvel.

Throughout the 19th century, the White House established itself as a place to show off America and greet the famous and the important. When Charles Dickens toured the U.S. in 1842, he found the home of President John Tyler a casual, respectful place, “and every man … appeared to feel that he was a part of the Institution, and was responsible for its preserving a becoming character, and appearing to the best advantage.” In 1860 the Japanese sent their first official emissaries to a Western culture. President James Buchanan gave the three ambassadors a tour of the house, and they attended a banquet featuring chicken, halibut, ham, ice cream, lobster, pigeon and turtle soup. Soon after, Buchanan welcomed Albert, Prince of Wales, who inspired a crush to see the future King Edward VII. Noted a local newspaper, “Confusion reigned. The Royal party have certainly seen Democracy unshackled.”

But the capital city the Founding Fathers positioned to unite the north and the south was soon endangered by the fissure of the Civil War. Kansas volunteers bivouacked in the East Room to protect Abraham Lincoln, and the president tested rifles on the grounds around the house. Then in July 1864 — half a century after the British burned the city — Lincoln used a spyglass to watch the movements of troops led by Confederate Lt. Gen. Jubal Early. Though Early found Washington “feebly manned,” he decided not to attack.

After the war, there had been small egg rolls at the White House and larger ones at the Capitol. But when Congress scoffed at the resulting cost of grounds upkeep, the Capitol rolls ceased. Some enterprising youngsters approached President Rutherford B. Hayes during his daily walk to see if he could host the famous hunt, and he and First Lady Lucy Hayes officially opened the White House lawn to celebrants. In 1879 Hayes received the home’s first telephone, and the following year Queen Victoria sent him an elegantly carved desk made from the timbers of the H.M.S. Resolute. It has since served nearly every president, and it became famous from the photo of John F. Kennedy Jr. playing underneath it as his father worked.

In the 1880s, the frugal Chester A. Arthur tried his own hand at redecorating. “Night after night,” recalled an aide, the widower president “would go from room to room and corridor to corridor, giving orders to change this and that according to his own taste.” When that didn’t work, he hired Louis Comfort Tiffany, and the designer grandly decorated the home.

Electric lights finally arrived in 1891 when Thomas Alva Edison’s electrician, Ike Hoover, installed fixtures for President Benjamin Harrison. Hoover went on to become the house’s chief usher, and he recalled how at the time it was still far from a showcase, with the ground floor “covered with damp and slimy brick; dust webs were everywhere.” First Lady Caroline Harrison worked to make it livable and sought to make additions on the west, east and south sides, but Congress instead gave her money to clean and fix up the place.

A real makeover came when Theodore Roosevelt arrived in 1901 with First Lady Edith Roosevelt and their six children. He officially called the mansion the White House, and he got Congress to approve $475,445 for construction, renovation and refurnishing. To orchestrate the work, the First Lady hired Charles McKim, one of the era’s great architects. McKim, a major exponent of Beaux-Arts classicism, stripped the Victorian decor, fixed the settling floors, excavated the basement and redecorated the State floor. In the process he turned the home into a Georgian showpiece. On the second floor, McKim took out the executive offices, which reeked of cigar smoke and were cluttered with spittoons, turning the whole story over to the First Family. He then constructed a West Wing for the commander in chief and his army of assistants, as well as an East Wing for guests.

A few years later, President William Howard Taft had the architect Nathan Wyeth enlarge the West Wing and install the famed Oval Office. All houses, though, need upkeep. The roof leaked, and in 1927 Calvin Coolidge had the architect William Adams Delano convert the attic into a full third floor. After a fire on Christmas Eve in 1929 gutted the West Wing, Herbert Hoover reconstructed the space, and when Franklin D. Roosevelt moved in, he had the wing rebuilt, adding a second floor and shifting the Oval Office to its current location.

The house’s constant alterations took their toll. Floors shifted, plaster sagged, and President Harry S. Truman wrote his sister, “The engineer said that the ceiling in the state dining room only stayed up from force of habit!” As Architectural Digest reported, “There is scarcely a beam in the entire building that has not been bored or cut through a dozen times.” Truman set out to rescue the home. “He sensed the importance of the White House was what it symbolized, meant and looked like, not the little details of things like light fixtures,” says historian [William] Seale. “He decreed that [the exterior] wall would not be disturbed, and the building rebuilt.”

Architect Lorenzo Winslow had workers pack up furniture and mantels and gut the inside. They then inserted a steel framework, created two new levels of sub-basement, installed a ventilation system, and reconstructed and fireproofed the interior. It took four years and four months to complete, and while the home looked essentially the same on the outside, Truman’s White House went from a structure with 62 rooms and 14 bathrooms to one with 132 rooms and 20 bathrooms.

When John and Jacqueline Kennedy took residence in 1961, the First Lady found a lot of the furnishings unworthy of the home. The position of White House curator was created, as was a Fine Arts Committee for the White House, and they searched for appropriate art and antiques. Tragically, their stay was cut short, and the work Jackie had planned was not completed when an assassin felled JFK in 1963.

Changes continued. First Lady Pat Nixon added 18 portraits of presidents and First Ladies to the home’s collection and brought in museum-quality furniture. More recently, President Barack Obama installed a beer-brewing kit in the kitchen. And with the introduction of beehives to the South Lawn, the home has extended the reach of Jefferson’s wine cellar with its own White House Honey Brown Ale, the first alcoholic beverage brewed or distilled on the White House grounds.

The house that Obama moved into is five times as large as the building Washington began 224 years ago. Topping out at 55,000 square feet, the home has six stories, 132 rooms, 35 bathrooms, a tennis court, a bowling alley, a swimming pool and a putting green. Even so, it still recalls the vision that Washington had for the home and office of the citizen president when he studied James Hoban’s architectural drawings. And despite its additions, subtractions and fires — scorch marks from Ross’s arson are still visible below the North Portico — the place has flourished and fulfilled Washington’s hopes. As First Lady Michelle Obama said in November 2015, “While our family might live upstairs, the president and I know that the White House is truly the People’s House. It belongs to all of us, and it should reflect and celebrate the very best in us.”

Read more in the TIME special edition Inside the White House, available on Amazon. Or, check out the full range of TIME special editions, including Barack Obama: Eight Years and the Donald Trump election special.