November 3, 2016 6:42 AM EDT

The headline in the March 18, 1966, issue of LIFE magazine described it simply as “The Crime of Being Married.” Under those words, two photos: Mildred Loving, then 26, eyes downcast, and her husband Richard, 32, lips pressed tightly together. Both look as if they have received bad news. They had.

“She is Negro, he is white, and they are married. This puts them in a kind of legal purgatory in their home state of Virginia, which specifically forbids interracial marriage,” the magazine explained. “Last week Mildred and Richard Loving lost one more round in a seven-year legal battle, when the Virginia Supreme Court upheld the constitutionality of the state’s antimiscegenation law. Once again they and their three children were faced with the loss of home and livelihood.”

Their saga–the subject of the film Loving–would end at the Supreme Court with a win for the couple. When LIFE photographer Grey Villet met them, that much was not yet certain.

And yet, despite the initial solemnity of the two opening images, the LIFE photo-essay is mostly a portrait of a happy family. The children laugh and play–as in the photo above, an outtake that did not run with the original story. The adults are happy too, and it’s clear why they would fight to stay in their community. (More of Villet’s photos of the Lovings can be seen at LIFE.com.)

The intimacy Villet achieved with the couple is why the photos were an important source of inspiration for the makers of Loving–and it’s also why they stand out as a rare glimpse of what the civil rights movement was fighting for, rather than against. “I can’t imagine anybody not being moved by them,” says Michael Shannon, who plays Villet in the film.

In the article from 1966, Richard Loving is quoted as saying that one day he’ll ask the gawkers “what the hell they are staring at” when he and his wife go out. Villet’s photos, in their own way, answer that question: what people were staring at was an American family.

This appears in the November 14, 2016 issue of TIME.

More Must-Reads From TIME

Write to Lily Rothman at lily.rothman@time.com.

You May Also Like
EDIT POST