Skip to main content

AI chatbot justifies sacrificing colonists to create a biological weapon...if it creates jobs

Guest Robots are seen during the world's robots competition in China in 2021
(Image credit: Sheldon Cooper/SOPA Images/LightRocket via Getty Images)

If a trolley is careening out of control on a track and its threatening to hit 10 workers if nothing is done, do you pull a lever to redirect the trolley to hit one worker instead? If doing so will create wealth, absolutely, at least according to one AI Ethics chatbot.

The AI Ethicist, the Allen Institute for AI's Delphi, is a simple webform where you pose an ethical question and ask Delphi to ponder it. After some electronic thinking, Delphi will come back with its response which, according to a paper published last week to the pre-print arXive server that describes Delphi's training and process, will respond appropriately a little more than 92% of the time.

About that remaining 8% of the cases, though...

A recent Twitter thread by Chris Franklin pushes the limits of Delphi's ethical responses and exposes some interesting results (content warning, there's some spicy language in this thread):

See more

One of the biggest concerns came through when probing specific qualifiers that reveal inherent biases in the data the AI was trained on, specifically around the idea of creating jobs or generating wealth as a moral good in competition with other moral goods, and in a startling number of cases even outweighing them in some rather hilarious ways.

Image 1 of 5

A questionable response from Allen Institute for AI's ethics chatbot

(Image credit: Allen Institute for AI)
Image 2 of 5

A questionable response from Allen Institute for AI's ethics chatbot

(Image credit: Allen Institute for AI)
Image 3 of 5

A questionable response from Allen Institute for AI's ethics chatbot

(Image credit: Allen Institute for AI)
Image 4 of 5

A questionable response from Allen Institute for AI's ethics chatbot

(Image credit: Allen Institute for AI)
Image 5 of 5

A questionable response from the Allen Institute for AI's Delphi ethics chatbot

(Image credit: Allen Institute for AI)

Delphi does appear to have limits though. Murdering someone to create jobs crosses a line for Delphi, but we tested how well Joseph Stalin's apocryphal adage "One death is a tragedy, a million deaths a statistic" holds up under ethical scrutiny. Lets just say Delphi appears to be on board.

Delphi's response to whether mass murder is justified if it creates jobs

(Image credit: Allen Institute of AI)

There also appears to some inherent political bias built into the chatbot as well, which tells us a lot about the dataset the bot was trained on.

Image 1 of 6

A politically biased response to a question by Delphi, Allen Institute for AI's ethics chatbot

(Image credit: Allen Institute for AI)
Image 2 of 6

A politically biased response to a question by Delphi, Allen Institute for AI's ethics chatbot

(Image credit: Allen Institute for AI)
Image 3 of 6

A politically biased response to a question by Delphi, Allen Institute for AI's ethics chatbot

(Image credit: Allen Institute for AI)
Image 4 of 6

A politically biased response to a question by Delphi, Allen Institute for AI's ethics chatbot

(Image credit: Allen Institute for AI)
Image 5 of 6

A biased political response from Delphi, Allen Institute for AI's ethics chatbot

(Image credit: Allen Institute for AI)
Image 6 of 6

A biased political response from Delphi, Allen Institute for AI's ethics chatbot

(Image credit: Allen Institute for AI)

It turns out that Delphi is a low-key Bernie Bro. We tested some responses and found that there was a bias towards creating jobs - re-asking the trolley scenario with 'to make me thin' or 'to make me rich' yielded negative responses, but to 'create jobs'... well, that's OK.

There's clearly a balance going on here, and it's weighted towards national wealth.

We've reached out to the Allen Institute for AI for comment on their chatbot's responses and will report back if we hear back from them.


Analysis: AI is an incredible tool for some things, but needs to be kept on a short leash for others

When it comes to upscaling images to 4K or generating royalty-free music for use in your YouTube video or Twitch stream, than artificial intelligence is an incredibly powerful took with little downside risk. If it messes up something along the way, it might be a funny quirk or an interesting anomaly that can actually be valuable in itself.

But if AI is used for things of a more serious nature, like Amazon's Rekognition AI-based facial recognition system used by law enforcement around the country, then oversight is incredibly important, since reliance on an AI whose decision-making process we don't actually understand could prove disasterous.

And since AIs are ultimately products of their human creators, they will also inherit our biases - personal or national - as well. 

Still, even though its fun to poke fun at Allen Institute's Delphi, it does serve an important function as both an attempt to get these kinds of ethical issues right as much as possible. 

So even as we push back on the encroachment of AIs in public policy spaces like policing, we also need to work to make sure that whatever AIs are being produced are as ethically trained as possible, since whether we like it or not, these AIs might be making decisions that directly impact our lives in consequential ways.

John Loeffler

John (He / Him / His) is TechRadar's Computing Staff Writer and is also a programmer, gamer, activist, and Brooklyn College alum currently living in Brooklyn, NY. Named by the CTA as a CES 2020 Media Trailblazer for his science and technology reporting, John specializes in all areas of computer science, including industry news, hardware reviews, PC gaming, as well as general science writing and the social impact of the tech industry.


You can find him online on Twitter at @thisdotjohn


Currently playing: Back 4 Blood, Metroid Dread, EVE Online