Some White Southerners Didn’t Want to Secede During the Civil War—Or Build Confederate Monuments After

This post is in partnership with the History News Network, the website that puts the news into historical perspective. A version of the article below was originally published at HNN.

“Ever since the Lee unveiling,” Elizabeth Van Lew of Richmond, Va., wrote to a Massachusetts friend in 1891, “I have felt that this was no place for me.”

This was a remarkable and revealing confession. Van Lew had long been at odds with the majority of her fellow white Richmonders: while they supported the Confederacy, she had stayed loyal to the Union and played a heroic role during the Civil War as the head of an interracial Federal espionage network in the rebel capital. The network helped Union prisoners of war escape the Confederacy and funneled military intelligence to General U.S. Grant. Van Lew skillfully stayed one step ahead of the Confederate authorities, aware that she risked banishment, imprisonment or death if caught.

Through all the trials of secession, war, and Reconstruction, her resolve to remain in Richmond had never wavered. But the May 29, 1890, unveiling of a massive equestrian statue of Robert E. Lee in Richmond — an event attended by a throng of more than 100,000 spectators — was the final straw. Van Lew knew that the ascendant cult of the Lost Cause, which posited a “solid South” of dauntless Confederates and loyal slaves, was designed to erase her and other anti-Confederate Southerners from the region’s public life and from its history books.

Recovering the history of Southern Unionism has been challenging work for modern historians, and it has yielded some bracing insights. William W. Freehling’s 2001 book The South vs. the South: How Anti-Confederate Southerners Shaped the Course of the Civil War revealed that 450,000 men from slave states fought for the Union army during the Civil War. The group of anti-Confederate Southerners with the most measurable impact on the war were the 150,000 black men who fought in United States Colored Troops regiments. They are part of the larger story of African American resistance to the Confederacy: slaves fled farms and plantations by the hundreds of thousands to seek refuge with the Union army and contributed to the Union victory not only as soldiers but as nurses, spies, scouts and in a host of other ways.

White Southern Unionists were more unevenly distributed across the slave states and their impact is harder to assess. Unionism prevailed in the four slaveholding border states (Maryland, Kentucky, Missouri and Delaware). There were pockets of Unionists even in the seceded states, especially in mountainous, “upcountry” districts where plantation slavery did not predominate — places like West Virginia and East Tennessee. But in the Confederacy’s plantation belts, white Unionists like Elizabeth Van Lew were a small and beleaguered minority. Their motivations varied widely. Some had family ties to the North; some advocated the economic modernization of the South; some had class-based resentments against the secessionist elite; some belonged to dissenting religious communities, such as the Quakers. Some, like Van Lew, embraced emancipation, but most were anti-secessionist rather than anti-slavery.

Modern scholarship has established that white Southern Unionists in the Confederacy had an impact disproportionate to their sparse numbers — by destroying Confederate assets, sowing dissension and aiding the Union army they exasperated Confederate authorities, who devoted time and resources to trying to roust them out. In a recent study of North Carolina, historian Barton Myers found that Unionists there furnished enough localized resistance to undermine Confederate control of roughly 1/3 of the state’s counties, forcing the Confederates to deploy resources, including vital manpower, to try to stabilize the contested areas.

White Southern Unionists played just as important a role in sustaining Northern morale as they did in bedeviling the Confederates. The very existence of loyalists such as Van Lew sustained the abiding Northern belief that substantial numbers of white Southerners yearned for deliverance from Confederate rule. This belief was rooted in the popular “Slave Power Conspiracy” theory that elite secessionists had bullied and duped the non-slaveholding white Southern masses into accepting disunion. Northerners went to war to save the South from the secessionists, and clung tenaciously to that war aim even in the face of massive evidence that Confederates did not want to be saved. The belief in deliverance was fueled by stories of dissent and disaffection, which were widely disseminated in Northern popular culture: stories of loyalist scouts, guides and spies lending their services to the Federal army; of refugee families accepting food from Union commissary stores; of teeming crowds welcoming Union armies of occupation; of white Southerners seeking amnesty by taking oaths of allegiance to the Union; of Confederate deserters returning like prodigal sons to the Union fold.

White Southern Unionists willing to endorse and defend Abraham Lincoln’s leadership and policies played a key role in this vision of deliverance, as a vanguard who could help lead other Southerners into the light. Men such as Cassius Clay of Kentucky, Andrew Jackson Hamilton of Texas, Edward Gantt of Arkansas and Andrew Johnson of Tennessee loomed large in Northern politics as symbols of Southern deliverance. As the head of the “Richmond Underground,” Van Lew too became such a symbol. As George Sharpe, Union chief of military intelligence for the Army of the Potomac, put it, “for a long, long time, [Van Lew] represented all that was left of the power of the U.S. government in the city of Richmond.”

When Federal troops entered Richmond in April 1865 Van Lew felt it a personal vindication. “Oh, army of my country,” she confided to her journal, “how glorious was your welcome!” But the postwar era brought further struggles and deep disappointments. Reconstruction exposed the fault lines among white Southern Unionists, pitting the small number of progressives such as Van Lew, who supported the Radical Republican program of black civil rights, against reactionaries such as President Andrew Johnson, who accepted emancipation but rejected black suffrage and racial equality. Van Lew had a season of influence during Congressional Reconstruction, when President U.S. Grant appointed her postmaster of Richmond, and she used that influence to promote civil rights and woman suffrage. But her political enemies gradually drove her out of office, charging that she was too radical, and mentally unstable. As ex-Confederates recaptured Southern politics, the complex story of the wartime divisions among Southern whites was overshadowed by the ritual glorification of the Lost Cause.

Van Lew’s life raised the question: Who counts as a Southerner? The stakes of the question were high. Defenders of slavery had long argued that the only “real” Southerners were those who upheld the dominant proslavery political orthodoxies of the region; anyone who dissented was by definition an outsider. Van Lew objected to the Lee statue because it sent an unmistakable message to Richmond’s African Americans and to their small number of white allies that they must remain on the margins of Southern politics. Van Lew’s 1891 lament about the Lee statue echoed that of John Mitchell, Jr., editor of the leading black newspaper in Virginia, the Richmond Planet. In a May 31, 1890, editorial on the Lee unveiling—entitled “What It Means”—Mitchell asserted that Confederate memorialization “fosters in this Republic, the spirit of Rebellion and will ultimately result in handing down a legacy of treason and blood.” This was a plea for justice and recognition for those Southerners who chose patriotism over treason during the Civil War, and progress over reaction in its aftermath. The South’s Confederate statues obscured the region’s political diversity, and still do.

Elizabeth Varon is Langbourne M. Williams Professor of American History at the University of Virginia, and author of Armies of Deliverance: A New History of the Civil War.