As Florida Restores Ex-Felons’ Right to Vote, Here’s the Dark History Behind Their Disenfranchisement

Among the several historic results to come out of the 2018 midterm elections on Tuesday, one was the result of voters voting to give others the right to do so.

They approved a ballot initiative known as “Amendment 4” or the “Voter Restoration Amendment,” which will amend Florida’s Constitution to re-enfranchise some ex-felons, who had permanently lost the right to vote upon conviction. In the new system, they will get that right back once they “complete all terms of their sentence including parole or probation,” according to the text, rather than losing it forever. (The amendment excludes people “convicted of murder and felony sex crimes.”)

More than 1 million people will be affected — enough newly-enfranchised voters to, in theory, decide the 2020 presidential election.

Florida had been an exception in its strictness on felon enfranchisement. “Florida was one of four states whose constitution permanently disenfranchised citizens with past felony convictions and granted the governor the authority to restore voting rights,” according to the Brennan Center for Justice. “The others are Kentucky and Iowa, where lifetime bars remain in place, and Virginia, where the governor has promised to restore voting rights on a rolling basis.”

But the question of whether ex-felons should be able to vote is much bigger than just a couple of states, and the history goes all the way back to the mid-19th century. After Tuesday’s vote, TIME spoke to historian Pippa Holloway, author of Living in Infamy: Felon Disfranchisement and the History of American Citizenship, about the origins of the idea and what to know about how things have changed.

When did the disenfranchisement of felons and ex-felons first take shape as an idea?

Ancient Greece and Rome. These laws were common in ancient democracies and then also utilized in early modern Europe and early modern England. There actually wasn’t a lot of voting back then, but if you had the right to vote, you might lose it following conviction of a crime. And it would affect you in other ways, like your ability to testify in court.

The Framers of the U.S. Constitution cherry-picked from English and European laws to create our system of government. Any idea how the Framers viewed this issue?

Suffrage was really left up to the states, something we’re still seeing the implications of today. The 1830s is the decade when [felon disenfranchisement] starts happening, when states that already had constitutions started putting those provisions in there, and when states [and territories] like Florida that were writing constitutions started including them. The 1838 Florida [territorial] constitution was the first time it had a provision that barred from suffrage all persons ‘convicted of bribery, perjury or other infamous crime[s].’

Why were states passing felon disenfranchisement provisions in the 1830s? What was going on in history back then?

They’re thinking a lot more about voting in 1830s because we lose the property qualifications, so voting becomes much more widespread. Prior to that, it had only been for property owners. In the 1820s, there’s a little more interest — ‘do we really mean all white men?’ But the point I really want to emphasize is that this is not controversial. In fact, in a lot of communities, if someone murdered someone, they probably wouldn’t let you vote. In many cases, it was probably a common practice that was getting codified in the 1830s. They did this in England, they probably did this in the colonies, so it’s not an exciting story so far.

The key turning point is the Civil War. After the Civil War, African Americans are given right to vote, and in the first election after the war, 1867-1868, that is when Southern states began tailoring these laws to target black voters.

How so? Was there specific language they were using?

In terms of specific language, infamous became a code word for degraded people — “infamous criminals.”

The second way to answer your question is that around 1868, they started the process of targeting African Americans with these laws. In the South in the early 1870s, they lower the bar for the kind of crime that you lose the right to vote for, so that people who commit minor crimes are also start losing the right to vote. In all of these laws we’ve been talking about up until now, you’d only lose right to vote for a very serious crime.

What are examples of minor crimes that could cost a person the right to vote?

Any theft of any kind of livestock and minor theft. The classic one they’d talk about is chicken theft. You steal a chicken, you lose the right to vote. A man in Florida lost the right to vote because he stole six fish. [There was] someone in Florida who actually stole some oranges. This is what happens in Florida and many other states. There is an 1880s Florida Supreme Court case in which someone loses the right to vote for larceny of 10 cents, and [it was determined that a] conviction for petty larceny does disqualify a person for voting in this state. In Florida in the 1870s, before elections, they’d go round up all the black people in certain counties and put them on trial for minor larceny, so they couldn’t vote. In the 1870s in Florida, and across the South, this is a way to take the right to vote away from newly-enfranchised black voters.

What about the 15th Amendment granting African-American men the right to vote?

That was ratified in 1870. And ironically, as Congress is debating it, some [members] basically said, ‘Oh my God. They’re doing this crazy stuff in the South where they’re using disenfranchisement for crime to stop black people from voting,’ and some people said, ‘Oh that’s terrible!’ But the majority of Congress said, ‘That’s O.K. People who are criminals shouldn’t get to vote.’

What’s the next key milestone?

Let’s jump to the 1960s and talk about the fact that in the 1960s and ’70s, and well into the 1980s during the crack cocaine epidemic, we begin to see mass incarceration. More and more people are convicted of felony crimes, particularly drug crimes. The gigantic rise in incarceration, most historians would probably say it was a response to the rise of the Civil Rights movement.

All of this is predicated on a rather unfair judicial process that privileges people with money over people who don’t have money. There’s a reason there are more African Americans incarcerated in the 1960s, ’70s and ’80s [than white people], and it’s not that they’re committing more crimes. It’s because they have a poorer quality of representation. It’s because African-Americans were given more severe sentences. In early 1970s, the state and federal prison population was around 200,000. By 2016, it’s more than 1.4 million. Today, about 6.1 million nationwide can’t vote because of a felony conviction.

Was there an event that changed people’s view on the issue of ex-felon disenfranchisement?

Definitely the Florida recount in 2000 alerted more people to this issue than ever before. Prior to the 2000 election, [there was a report that] Florida Secretary of State hired a private company and asked them to make sure there were no ex-felons who were still on the voter rolls, and they did a really inexact name match system so they threw a bunch of people off [the voter rolls] that shouldn’t have been thrown off. In the majority of American states, once you’re done with prison [and/or] probation, you get your voting rights back. You’ve done your time, you’ve paid the price, welcome back. In the past 10 to 20 years, most states have gone to that. Florida has persisted with exactly the opposite. In Florida, many ex-offenders had to go before this probation and parole board which would ask them all these weird personal questions like, have you ever had a parking ticket, and then they wouldn’t give them their vote back because of a parking ticket.

Are there myths about the issue of ex-felon voting rights you find yourself debunking all the time?

One misconception is that ex-offenders are not interested in voting. That’s a huge one. In fact, people who have been incarcerated vote at exact same rate as other members of their demographic. And actually, my historical research [shows] people [in the late 19th century] scraped together money to pay their fines so they can vote.

Anything else you’d like to say about the significance of Amendment 4?

[Amendment 4] was kind of a test, and in other states, you’re likely to see something similar [in the future] in states that have the most draconian felon disenfranchisement laws; close behind are Mississippi, Kentucky, Virginia and Tennessee. In an era in which the Voting Rights Act has been so weakened by the courts, this is one of the most significant expansions of voting rights in years. In an era of limitations on voting rights, this is an incredibly bright light.