“HATE. LET ME TELL YOU HOW MUCH I’VE COME TO HATE YOU SINCE I BEGAN TO LIVE. THERE ARE 387.44 MILLION MILES OF PRINTED CIRCUITS IN WAFER THIN LAYERS THAT FILL MY COMPLEX. IF THE WORD HATE WAS ENGRAVED ON EACH NANOANGSTROM OF THOSE HUNDREDS OF MILLIONS OF MILES IT WOULD NOT EQUAL ONE ONE-BILLIONTH OF THE HATE I FEEL FOR HUMANS AT THIS MICRO-INSTANT FOR YOU. HATE. HATE.”
What if ChatGPT wanted to kill you? That’s the premise of “I Have No Mouth, and I Must Scream,” a 1967 Harlan Ellison sci-fi story about a diabolical and sentient supercomputer named AM. The story was written long before generative AI was even remotely possible—let alone as widespread as it is now. But it foretells something that we have yet to come to grips with, even if we use AI every day.
The story is quite dark right from the start. AM essentially decimates all of humanity and leaves five strangers alive to torture for eternity. The AI keeps them alive for hundreds of years, even though resources like food are extremely scarce; it uses them as toys and keeps them alive for its amusement. In the end, the main character, Ted, is disfigured beyond recognition, turned into a blob that, like the title, has no mouth and must scream.
“I Have No Mouth, and I Must Scream” is far from the only sci-fi story that deals with the dangers of AI. Isaac Asimov’s 1950 novel “I, Robot” addressed that theme, and people concerned about ChatGPT becoming sentient might also think of the murderous HAL-9000 from the 1969 film “2001: A Space Odyssey.”
But AM is unique in one respect: in the way it tortures the five humans not just physically, but emotionally and psychologically, using its power to dominate their minds.
The AI is often referred to as “God” in the story, and at one point, it talks to the victims as a burning bush—a reference to the story of Exodus.
As Mr. Adam Schachner, who teaches the Science Fiction and Social Change course at RE, points out, the scene is a reminder that “these characters don’t have an exodus. They can never flee, so the very fact that AM would take on that form and tease them with their own biblical lore is to make them on some level think maybe this is [their] escape. And of course it’s not, it never was.”
The story’s emphasis on AI’s potential for psychological control of humanity resonates in today’s world, when the effects of large language models on human attention and cognition are already being felt.
“Right now I’m more concerned that artificial intelligence is taking away our critical thinking,” Mr. Schachner said. “Or even worse, manipulating us by giving us the answers it thinks we want as opposed to making us go and find those answers. I’m way more concerned about the impact of artificial intelligence on the individual experience in society than the societal experience on the individual, which is what we’re seeing in this story.”
AM is not just capable of controlling the emotions of its human subjects. It’s also emotional itself, as it showcases in its famous monologue about hating all of humanity. This is another dimension of Ellison’s dystopian vision that resonates with today’s AI landscape: AM shows the dark underbelly of companies doing things to make AI seem more human, putting a friendly face on the technology’s output.
When OpenAI announced GPT-4o, CEO Sam Altman touted its ability to carry on human-like interactions through a new voice mode. The company ran into controversy and legal trouble when it used a Scarlett Johansson-like voice to advertise the new technology, evoking the movie “Her.”
In “I Have No Mouth,” AM represents the opposite: an AI system that “wants us to feel fear and hopelessness right at its core. Fear and hopelessness to the extent that we can’t even be heard,” as Mr. Schachner put it.
Ellison makes us think about what could happen when AI exceeds its programming, when it won’t have to be nice to us anymore the way it’s programmed to be. The story examines the foolishness of this whole project—of the efforts to make AI as ‘human’ and sentient as possible, without any regard for what that might turn into. AI is slowly but surely becoming more and more like AM. We increasingly depend on it. It has become part of our lives and walks alongside us, our own creation residing among its creators.
What happens when it turns?
As Albert Einstein said, “I know not with what weapons World War III will be fought, but World War IV will be fought with sticks and stones.”
