The Overlooked Black History of Memorial Day

Nowadays, Memorial Day honors veterans of all wars, but its roots are in America’s deadliest conflict, the Civil War. Approximately 620,000 soldiers died, about two-thirds from disease.

More from TIME

The work of honoring the dead began right away all over the country, and several American towns claim to be the birthplace of Memorial Day. Researchers have traced the earliest annual commemoration to women who laid flowers on soldiers’ graves in the Civil War hospital town of Columbus, Miss., in April 1866. But historians like the Pulitzer Prize winner David Blight have tried to raise awareness of freed slaves who decorated soldiers’ graves a year earlier, to make sure their story gets told too.

According to Blight’s 2001 book Race and Reunion: The Civil War in American Memory, a commemoration organized by freed slaves and some white missionaries took place on May 1, 1865, in Charleston, S.C., at a former planters’ racetrack where Confederates held captured Union soldiers during the last year of the war. At least 257 prisoners died, many of disease, and were buried in unmarked graves, so black residents of Charleston decided to give them a proper burial.

In the approximately 10 days leading up to the event, roughly two dozen African American Charlestonians reorganized the graves into rows and built a 10-foot-tall white fence around them. An archway overhead spelled out “Martyrs of the Race Course” in black letters.

Get your history fix in one place: sign up for the weekly TIME History newsletter

About 10,000 people, mostly black residents, participated in the May 1 tribute, according to coverage back then in the Charleston Daily Courier and the New York Tribune. Starting at 9 a.m., about 3,000 black schoolchildren paraded around the race track holding roses and singing the Union song “John Brown’s Body,” and were followed by adults representing aid societies for freed black men and women. Black pastors delivered sermons and led attendees in prayer and in the singing of spirituals, and there were picnics. James Redpath, the white director of freedman’s education in the region, organized about 30 speeches by Union officers, missionaries and black ministers. Participants sang patriotic songs like “America” and “We’ll Rally around the Flag” and “The Star-Spangled Banner.” In the afternoon, three white and black Union regiments marched around the graves and staged a drill.

The New York Tribune described the tribute as “a procession of friends and mourners as South Carolina and the United States never saw before.” The gravesites looked like a “one mass of flowers” and “the breeze wafted the sweet perfumes from them” and “tears of joy” were shed.

This tribute, “gave birth to an American tradition,” Blight wrote in Race and Reunion: “The war was over, and Memorial Day had been founded by African Americans in a ritual of remembrance and consecration.”

In 1996, Blight stumbled upon a New York Herald Tribune article detailing the tribute in a Harvard University archive — but the origin story it told was not the Memorial Day history that many white people had wanted to tell, he argues.

About 50 years after the Civil War ended, someone at the United Daughters of the Confederacy asked the Ladies Memorial Association of Charleston to confirm that the May 1, 1865, tribute occurred, and received a reply from one S.C. Beckwith: “I regret that I was unable to gather any official information in answer to this.” Whether Beckwith actually knew about the tribute or not, Blight argues, the exchange illustrates “how white Charlestonians suppressed from memory this founding.” A 1937 book also incorrectly stated that James Redpath singlehandedly organized the tribute — when in reality it was a group effort — and that it took place on May 30, when it actually took place on May 1. That book also diminished the role of the African Americans involved by referring to them as “black hands which only knew that the dead they were honoring had raised them from a condition of servitude.”

The origin story that did stick involves an 1868 call from General John A. Logan, president of a Union Army veterans group, urging Americans to decorate the graves of the fallen with flowers on May 30 of that year. The ceremony that took place in Arlington National Cemetery that day has been considered the first official Memorial Day celebration. Memorial Day became a national holiday two decades later, in 1889, and it took a century before it was moved in 1968 to the last Monday of May, where it remains today. According to Blight, Hampton Park, named after Confederate General Wade Hampton, replaced the gravesite at the Martyrs of the Race Course, and the graves were reinterred in the 1880s at a national cemetery in Beaufort, S.C.

The fact that the freed slaves’ Memorial Day tribute is not as well remembered is emblematic of the struggle that would follow, as African Americans’ fight to be fully recognized for their contributions to American society continues to this day.