Iason Gabriel

Iason Gabriel is a rare sight at a big tech company: a political theorist with a Ph.D. from the University of Oxford. He’s Google DeepMind’s point-man on the philosophical underpinnings of AI “alignment,” or the question of how to define the ethical values to which AI should be held. And in April, he co-authored a 274-page paper on the ethics of AI agents—bots that can take actions in the real world, which many AI companies believe are just around the corner. The paper was perhaps the most comprehensive look yet from inside a leading AI company at the constraints that should be placed on more autonomous forms of AI, and how companies should balance the sometimes competing demands of themselves, their users, and society at large. It is among the first steps, Gabriel says, in Google’s effort to anticipate how AI agents might change the world, and it will influence how the company designs both its new AI systems and the rules that govern them.

Among the paper’s recommendations: AI companies should avoid making their AI agents too human-like, and that agents should always disclose that they are AIs rather than humans. Tech companies must do all they can to not exploit the trust placed in them by users who might disclose to agents their sensitive personal data, the paper says. And AI companies must carefully consider what actions their agents should, firstly, be able to take automatically; secondly, require human confirmation for; and, thirdly, be prohibited from doing entirely.

The work of Gabriel and his colleagues has the potential to set the tone for how Google, which has billions of users worldwide, rolls out a potentially transformative new technology. But Google has gone against its own warnings before. In 1998, a paper published by Google’s co-founders Larry Page and Sergey Brin warned that any future advertising-funded search engines would “be inherently biased towards the advertisers and away from the needs of consumers.” The company now makes most of its money from advertising.