• Tech
  • Innovation

‘I Don’t Have Ethics’: The Dark Heart of Google’s Latest Chatbot

2 minute read

A new research paper released by Google this week reaffirms what’s been more or less obvious since HAL 9000 murdered four astronauts in Stanley Kubrick’s 1968 film 2001: A Space Odyssey: computers do not make particularly dependable friends.

The Google project—an admittedly more benign exercise than Kubrick’s—was yet another attempt to fashion a “conversational model” of artificial intelligence. Colloquially, we refer to these systems as “chatbots”—programs that can somewhat convincingly engage in human discourse.

The researchers behind the experiment, Oriol Vinyals and Quoc V. Le, programmed their particular chatbot with two sets of data: guidelines from an IT troubleshooting help desk and a database of movie subtitles. The ensuing dialogue was mostly awkward and occasionally nihilistic.

“What is the color of water?” the researchers asked.

“Water,” the computer replied simply.

“Where are you now?” they asked.

“I’m in the middle of nowhere.”

“Tell me the definition of morality,” the researchers requested.

“I don’t have ethics,” the machine advised them.

According to the transcript, this went on for awhile. Philosophical banter proving futile, Vinyals and Le switched gears, with more coherent results.

“What do you think about Cleopatra?” they asked.

“Oh, she’s very regal.”

Read more dialogue here.

These Vintage Computer Ads Show We've Come a Long, Long Way

http://pop.bitpig.com/oldads/nov/pdp-11-70.jpg
1974
1976
1976
http://www.everyjoe.com/2009/07/30/technology/blast-from-the-past-the-3398-10mb-hard-disk/
1977
http://www.mopo.ca/uploaded_images/honeywell_email-763551.jpg
1977
http://www.vintagecomputing.com/wp-content/images/retroscan/ibm5110_large.jpg
1978
1978
1980
http://www.macmothership.com/gallery/MiscAds/AdamAd.JPG
1980
http://www.vintagecomputing.com/wp-content/images/retroscan/ibm_pc_woman_large.jpg
1981
http://www.aresluna.org/attached/computerhistory/ads/international/lotus/pics/byte8311
1983
http://www.vintagecomputing.com/wp-content/images/retroscan/msmouse_large.jpg
1983
http://www.vintagecomputing.com/wp-content/images/retroscan/osborne_large.jpg
1983
http://pop.bitpig.com/oldads/nov/keeping-up.jpg
1984

More Must-Reads from TIME

Contact us at letters@time.com