The aliens have landed, and we created them

IT IS NOT every day that I read a prediction of doom as arresting as ’s in Time magazine last week. “The most likely result of building a superhumanly smart AI, under anything remotely like the current circumstances,” he wrote, “is that literally everyone on Earth will die. Not as in ‘maybe possibly some remote chance,’ but as in ‘that is the obvious thing that would happen.’ … If somebody builds a too-powerful AI, under present conditions, I expect that every single member of the human species and all biological life on Earth dies shortly thereafter.”

Do I have your attention now?

Yudkowsky is not some random Cassandra. He leads the , a nonprofit in Berkeley, California, and has already written extensively on the question of artificial intelligence. I still remember vividly, when I was researching my book Doom, his warning that someone might unwittingly create an AI that turns against us — “for example,” I suggested, “because we tell it to halt climate change and it concludes that annihilating Homo sapiens is the optimal solution.” It was Yudkowsky who some years ago proposed a modified Moore’s law: Every 18 months, the minimum IQ necessary to destroy the world drops by one point.

Now Yudkowsky has gone further. He believes we are fast approaching a fatal conjuncture, in which we create an AI more intelligent than us, which “does not do what we want, and does not care for us nor for sentient life in general. … The likely result of humanity facing down an opposed superhuman intelligence is a total loss.”

He is suggesting that such an AI could easily escape from the internet “to build artificial life forms,” in effect waging biological warfare on us. His recommendation is clear. We need a complete, global moratorium on the development of AI.

This goes much further than the signed by Elon Musk, Steve Wozniak (the Apple co-founder), and more than 15,000 other luminaries that calls for a six-month pause in the development of AIs more powerful than the current state of the art. But their motivation is the same as Yudkowsky’s: the belief that developing AI with superhuman capabilities in the absence of any international regulatory framework risks catastrophe. The only real difference is that Yudkowsky doubts that such a framework can be devised inside half a year. He is almost certainly right about that.

The obvious analogy is with two previous fields of potentially lethal scientific research: nuclear weapons and biological warfare. We knew from very early in the history of these fields that the potential for catastrophe was enormous — if not the extinction of humanity, then at least death on a vast scale. Yet the efforts to curb the proliferation of nuclear and biological weapons took much longer than six months and were only partly successful. In 1946, the US proposed the Baruch Plan to internationalize nuclear research. But the Soviet Union rejected it and there was soon a frenetic nuclear arms race. The most that was achieved was to limit the number of countries that possessed nuclear weapons (through the Non-Proliferation Treaty, which came into force in 1970) and to slow down and eventually reverse the growth of superpower arsenals.

Similarly, the Biological Weapons Convention that came into force in 1975 did not wholly end research into such weapons. The Soviets never desisted. And we know that all kinds of very hazardous biological research goes on in China and elsewhere, including the gain-of-function experiments with coronaviruses, which it seems increasingly to the COVID-19 pandemic.

So if Yudkowsky is right that AI is potentially as dangerous as nuclear or biological weapons, a six-month pause is unlikely to achieve much. On the other hand, his call for a complete freeze on research and development has about as much chance of success as the Baruch Plan.

One obvious difference between those older deadly weapons and AI is that most research on AI is being done by the private sector. According to the of the Stanford Institute for Human-Centered AI, global private investment in artificial intelligence totaled $92 billion in 2022, of which more than half was in the US. A total of 32 significant machine-learning models were produced by private companies, compared to just three produced by academic institutions. Good luck turning all that off.

But is the analogy with what we used to call “The Bomb” correct? That depends on your taste in science fiction. Just about everyone has heard of Skynet, which originated in the 1984 film The Terminator, starring a young Arnold Schwarzenegger. For younger readers, the premise is that “Skynet,” a computer defense system “built for SAC-NORAD by Cyber Dynamics,” goes rogue in the future and attempts to wipe out humanity with a nuclear attack. John Connor leads the human resistance to Skynet and its robot Terminators. Skynet responds by sending Terminators back in time — because of course time travel is easy if you’re a really powerful AI — to kill Connor’s mother.

Yet there are many other versions of AI in science fiction. For example, in Ted Chiang’s (2010), AI manifests itself as “digients” — initially harmless and helpless computer-generated pets and companions, a little like baby chimpanzees. They spend quite a long time learning to be intelligent. In this version of the world, the moral problem is that we humans are tempted to exploit the digients as robot slaves or sex toys.

In essence, Yudkowsky’s numerous critics want us to believe that AI is more digient than Skynet. Writing on Twitter, , founder of the machine-tool firm GenFab, accused Yudkowsky “and the other hardline anti-AI cultists” of being “out of their depth, both in terms of command of basic technical elements of this field but also in terms of their emotional states. … Many things are coming, Skynet is not one of them.” Shutting down AI research, argued Parlmer, would deprive sick people of potential breakthroughs in medical science.

, the CEO of the Atlantic, agreed that Yudkowsky and other Luddites were overstating the risks. “I recently made a children’s book for my nine-year-old’s birthday using Dall-E and GPT-4 about a World Cup between his stuffed animals,” he told Atlantic staff. “The bears won and he loved it. … Let’s all build in some time to experiment. We’ll make cool stuff and we’ll learn while we do it.”

My Bloomberg Opinion colleague was more pragmatic. He posed some hypothetical questions: “What if, in 2006, we had collectively decided to suspend the development of social media for six months while we pondered possible harms from its widespread use? Its effects were hardly obvious at the time, and they are still contested. In the meantime, after the six-month delay, how much further along would we have been in the evaluation process? And even if American companies institute a six-month pause, who’s to say that Chinese companies will?”

But the most eloquent defender of unrestrained AI research and development is my old friend Reid Hoffman, the founder of LinkedIn, who has written on the subject … approximately half of which was generated by AI.

For the lay reader, the problem with this debate is twofold. First, the defenders of AI all seem to be quite heavily invested in AI. Second, they mostly acknowledge that there is at least some risk in developing AIs with intelligence superior to ours. Hoffman’s bottom line seems to be: Trust us to do this ethically, because if you restrain us, the bad guys will be the ones who do the development and then you may get Skynet.

So let me offer a disinterested view. I have zero skin in this game. I have no investments in AI, nor does it threaten my livelihood. Sure, the most recent large language models can generate passable journalism, but journalism is my hobby. The AI doesn’t yet exist that could write a better than I can, not least because a very large number of the relevant historical documents are not machine-readable.

(To be continued.)

BLOOMBERG OPINION