Abeba Birhane

A cognitive scientist by training, Abeba Birhane started on the path toward AI research when she noticed a vital task that almost nobody was doing. AI models were being trained on larger and larger datasets—collections of text and images—gathered from the internet, including from its darkest corners. But Birhane realized that as those datasets were ballooning from millions to billions of individual pieces of data, few people were systematically checking them for harmful material, which can result in AIs becoming structurally racist, sexist, and otherwise biased.

With a small group of fellow researchers, Birhane has forged the beginnings of a new discipline: auditing publicly accessible AI training datasets. The work can be taxing. “Most of the time, my screen is not safe for work,” says Birhane, who is now a senior adviser in AI Accountability at the Mozilla Foundation and an adjunct assistant professor at Trinity College Dublin. “I used to love working in a café; now I can’t.”

In a recent paper, currently under peer review, Birhane and her co-authors come to a startling conclusion: AI models trained on larger datasets are more likely to display harmful biases and stereotypes. “We wanted to test the hypothesis that as you scale up, your problems disappear,” Birhane says. Their research indicated the reverse is true. “We found that as datasets scale, hateful content also scales.” (This interview has been condensed and edited for clarity.)

TIME: What are the implications of your findings for the wider AI industry?

Abeba Birhane: We should take the claim that scale makes everything better with a big bucket of salt. Because what our research has shown is that scale actually makes things worse, if what you are aiming for is a just, equitable, and fair model. Focusing on something manageable, on a relatively small scale, is much better. If it’s smaller, you have a better chance of knowing what’s in the dataset and how various concepts are represented. When a dataset consists of billions of pieces of data, it’s chaos. It’s just impossible to delve in and look at it, to diagnose the problems and find solutions.

One of the ways the bigger AI companies are getting around this problem is to build supplementary systems on top: building content-moderation classifiers to filter harmful material from their datasets, and using techniques like reinforcement learning to incentivize harmless model behavior. Does that strike you as an appropriate solution to the issues your research has raised?

It’s not a very sustainable model. It comes at a huge cost to disenfranchised underpaid workers, often in the so-called third world. That’s not a good solution—not for the people that have to suffer and pay for it. To find a solution, you have to first know your problem. There is not enough effort put into understanding problems that persist within datasets. And because corporations like OpenAI tend to be completely closed, we really don’t know how they source their dataset, how they detoxify their dataset. So when you have very little information about the kind of processes they follow, it’s difficult to suggest a solution. For me, the initial step is to open up.

What made you want to do this type of research?

To be honest, I was never excited about this work. I’m a cognitive scientist by training, but the department where I did my Ph.D. was run by the school of computer science. So I was working in a lab full of machine-learning researchers, and I was intrigued by how they sourced their data, how much attention—or lack thereof—they paid to the data. I saw that datasets are really critical to model performance, but also, glaringly, that not many people were paying enough attention to them. Sometimes I get really frustrated, because when you repeatedly look at really gruesome images, it makes you feel sh-tty. It’s not nice. So I guess I was almost forced into it, because nobody else was doing it. As opposed to running with happiness to do this kind of work.

Write to Billy Perrigo at billy.perrigo@time.com.