See Photos From the ‘Lonely’ Childhood That Shaped Princess Diana

Before Diana was the Princess of Wales, she was Diana Spencer. And while contemporary audiences may remember Diana — who died 20 years ago this Thursday, at 36, in a car crash in Paris — for braving the paparazzi, in her early life she was known for being the shy and quiet one in her family.

The above photos shed light on her early life, from scenes at her birthplace, Park House, to family summer vacations in Sussex. As TIME described her privileged upbringing in an April 20, 1981, cover package published shortly after the then-19-year-old Diana became engaged to the Prince of Wales Charles, the house that the Spencer family rented from the Crown had the only heated pool in the area — and that pool was more than a symbol of wealth. Because Park House was part of the royal estate at Sandringham, the neighbors who came by to take advantage of the pool included the Windsor children. “Diana and Charles may have met a few times when she was a child and he already a young man, but everyone is fuzzy on the details,” the story noted.

That pool was not the only indicator of her privilege, the story continued:

But, despite having all the makings of a happy childhood, that period could also be a solitary one for the young Diana. She had a strong connection with animals, doting on her Shetland pony and serving as head of “Pets’ Corners” at her boarding school Riddlesworth Hall, but often spent time without many people around. Her parents’ marriage having fallen apart before she was 10, the family home Althorp House could be “a lonely place” for Diana, Tina Brown, the former editor of The New Yorker and The Daily Beast, writes in the forward of the new book Remembering Diana: A Life in Photographs.

“A frequent erroneous press assumption,” she writes, “was that her distinguished aristocratic background meant familiarity with a grand, high-society world. In fact, Diana’s childhood was limited in its circle and almost feral in its neglect. She spent most of her free time with the servants below stairs (one reason, perhaps, she could communicate so well with ordinary people).”