A Brief History of the Kentucky Derby Hat

“They’r-r-re OFF!” wrote TIME in a 1926 recap of the Kentucky Derby.

“The long roar thundered like a wave, grumbled like a rising sea-surge through the crowd down the long stretch,” the piece continued. “The stands seemed to sway, to swell with it; hats and parasols and a foam of faces rose, hesitated for an instant on the top of the wave, settled slowly down into a whisperless silence. The horses moved down the stretch. It was a perfect start.”

The piece, “In Louisville,” contained TIME’s first-ever reference to Kentucky Derby hats—but at the time, the hats were hardly remarkable. Their presence was mentioned as a given. And one look at the vintage Kentucky Derby photos above will reveal that such a situation continued well into the race’s history: many women and men wore hats, but they weren’t the outrageous (and often ridiculous) hats for which the race is known today.

Now, however, the grandstand at Churchill Downs is one of the rare places in America where elaborate headwear is the norm, and at the race this Saturday the hats are sure to be a focal point. How did that happen?

Get your history fix in one place: sign up for the weekly TIME History newsletter

Fashion has always been an important part of the Kentucky Derby. It was after traveling to the famed Derby races in England and the Grand Prix de Paris in France in 1872, that Col. Meriwether Lewis Clark Jr. decided to establish a similar high-profile horse race in America. He raised money for a racetrack outside of Louisville, Ky., and held the first Kentucky Derby in 1875.

Though races were a mainstay for British and French society, American women at the time might have hesitated to stay away from horse racing, and the gambling and drinking that went along with it. But if wouldn’t do for the new race to seem seedy. So, in pursuit of his vision, Clark and his wife enlisted the ladies of Louisville to attend the races to picnic with friends. They knew that part of creating allure for the event would be positioning it as a fashion event — so the dress code required “full morning dress” for men and women from the start.

Kentucky Derby Fashion Inspiration From 1945

The picnicking women therefore wore hats and gloves with their dresses. And though the attire has evolved somewhat throughout the decades, the hats have remained a constant. By the 1920s, though the daytime Kentucky event didn’t attract much of the flapper style for which the era is remembered, the ladies could choose between formal suits or dresses to go with a range of fashionable hat styles.

In the 1960s, attendance and fashion rules relaxed a bit across the nation, and at the racetracks. As hats receded from the category of everyday clothing, the crucial change occurred: people who didn’t wear hats all the time were more likely to use the excuse to wear something extreme, with bigger brims and bolder hues. Going all out became a way to gain attention and admiration.

How To Throw a 1930s-Inspired Mint Julep Party for the Kentucky Derby

Since then, the hats have become fodder for chatter and style watches alike. The Kentucky Derby website claims that the fashion at the Royal Wedding of Prince William and Kate Middleton renewed enthusiasm about hats, fascinators and headpieces that defy categorization. This year marks the 142nd time people will arrive at Churchill Downs under the weight and shade of their derby hats—but even Col. Clark himself could not have guessed how important the headwear itself would become to spectators of a horse race.