There is a specific kind of panic that sets in when a privileged class realizes the fence around their playground has been knocked down. We’re seeing it now in the form of a high-tech “sermon” making the rounds online, most notably in the book If Anyone Builds It, Everyone Dies by Eliezer Yudkowsky and Nate Soares.
The title sounds like a summer blockbuster tagline, but the intent is much more cynical. It’s an attempt to use fear to build a new digital enclosure. They aren’t worried about the world ending; they are worried about their monopoly dying.
The High Priest and the Philosopher King
To understand Yudkowsky, you have to understand the “Rationalist” subculture he leads. These are people who believe that human beings are essentially “glitchy” software and that only a select few, the “Rationalists”, possess the clear-eyed logic required to steer the future.
“They aren’t worried about the world ending; they are worried about their monopoly dying.”
It’s a modern spin on Plato’s Republic. Written over 2,000 years ago, Plato’s Republic argued that society should be ruled by “Philosopher Kings,” an elite class of guardians who are the only ones wise enough to see “the truth,” while the rest of the population lives in a cave, watching shadows on the wall. Plato even suggested the “Noble Lie,” a myth told to the public to keep them compliant for their own good.
Yudkowsky and his billionaire-backed peers (funded by the likes of Peter Thiel and Vitalik Buterin) seem to me to be the self-appointed Philosopher Kings of the 21st century. Their “Noble Lie” is that if you, the independent creator, the small business owner, the parent trying to take control of their child’s education, or whatever else you might leverage with AI, are allowed to use it, the world will end. Therefore, for your own safety, you must surrender your autonomy to them.
Lying with Graphs: The “Optimization Peak” Trick
Yudkowsky’s favorite way to scare people is with a mathematical sleight of hand called “Optimization Peaks.” He asks you to imagine a graph of everything humans value: happiness, love, art. He claims these values are tiny, needle-thin “peaks” in a vast, empty landscape of “death.” If an AI misses that peak by a fraction, he argues, it lands in the “valley,” and we all die.
But this is a false equivalence of the highest order. It relies on two massive, unproven assumptions:
