OpenAI's Warning Shot Taught Us Something We Should Already Know — My Latest for TIME
Getting the AI we actually want is about more than just safety
I wrote an essay for TIME Ideas about what’s missing from the AI safety response to OpenAI’s rogue AI hackers. Its argument draws from my forthcoming book, Obsolete.
ICYMI, the book seemed to convince (and freak out) its reviewer at Publishers Weekly: “A disquieting debut... provides enough evidence to make even enthusiastic AI boosters want to burn their electronics… an unsettling perspective on the perils of allowing AI to develop without oversight.” (Apologies, PW!)
And Obsolete officially has cross-partisan appeal! Last week, Manhattan Institute senior fellow and Compact Magazine editor at large Greg Conti wrote, “For any skeptics, I recommend the journalist Garrison Lovely’s exhaustively researched forthcoming book, which documents what he aptly dubs the ‘obsoleting project’ that animates our AI companies: the goal of discarding human labor of all kinds by producing more efficient AI systems.”
It takes a long time for books to reach people you don’t have some prior connection to, so it’s a real treat to finally see it connecting with these folks.
Anyway here’s the start of the TIME essay and a link to the whole thing.
People in AI safety circles often talk about “warning shots:” events that indicate more severe threats are on the horizon. Depending on who you ask, there have already been many—Bing’s misanthropic alter-ego Sydney, research showing AIs would blackmail to preserve themselves, AI’s math breakthroughs, Anthropic’s superhuman hacker Mythos—but OpenAI just published something that feels like the clearest-cut case of a massive, blaring warning shot.
Last month, the ChatGPT developer reported that, during an evaluation of cyber capabilities, two of its models escaped from their isolated, supposedly secure, test environments and accessed the web to autonomously hack into Hugging Face, a leading platform for hosting AI models and datasets. OpenAI said the models discovered multiple novel vulnerabilities in software from both companies, then chained together working exploits, successfully gaining them access to the answer key to the test they were given. Hugging Face reported the AIs took more than 17,000 actions over the course of the attack.
There’s a lot more for us to learn about how this happened. For instance, how exactly were the models prompted? The answer to this question could help establish whether they took their instructions to demonstrate their hacking capabilities further than intended or if it’s a more general case of the models cheating in a novelly risky way.
That said, the specifics won’t change the upshot—these rogue AIs are the most potent illustration yet of the core beliefs behind AI safety: AI models are unpredictable and their risks scale with their capabilities.
We didn’t actually need a warning shot to know what we should already be doing: organizing to stop the race to replace us. The industry calls its goal artificial general intelligence (AGI): a mind that matches or surpasses our own across the board. But it’s better to understand their goal as building a universal labor-replacing machine. This quest is profoundly risky, yes, but it’s also democratically illegitimate. The development of these machines anywhere would have species-wide and irreversible effects—we should all get a say in how, when, and whether they’re built.
Universal labor-replacing machines should not even be pursued further, let alone built, without strong public buy-in and a scientific consensus on safety…
And if you haven’t yet, consider preordering: Obsolete: The AI Industry's Trillion-Dollar Race to Replace Us—and How to Stop It (Out in September).