In a BusinessWorld opinion column, independent curator and critic Marian Pastor Roces examines the widening between AI alarmism and complacency, drawing on Naomi Klein's latest book and a recent episode involving autonomous AI agents.

The world's most careful thinkers, she writes, take seriously the possibility — put at roughly 10% — that rogue AI could cause human extinction before the decade is over. Yet most of them also guard against a Biblical apocalyptic tenor. Between the extremely credible possibility of End Times, on one hand, and dismissal of the ultimate threat on the other, lies a space that needs veteran critics to figure out. Finding what is wrong with the doomsayers — while actually believing some of them — is a test of stamina. It means understanding the technology, the inventors and makers, the public good, those who wish to define the future for everyone else, the clueless and those who know too much, the history of apocalyptic talk, its pertinence to the climate crises, and posthuman scenarios. It is a bit much.

The best course is to use one's wits to choose an Explainer — someone who has thought about all of it for a very long time and written brilliantly. That would be Naomi Klein. Among very few others, unfortunately for humanity.

An Explainer: Naomi Klein

Klein, the activist, journalist, and filmmaker who famously critiqued the history of neoliberalism and works for climate justice, is a regular on lists of the world's most influential thinkers. Her latest book, written with fellow activist Astra Taylor, is End Times Fascism: The Fight for the Living World.

To Klein, end-times fascism is old fascism played out today with one crucial difference: there is "no future horizon." Technology's über-actors — Elon Musk, Sam Altman, and Mark Zuckerberg, among the literal handful of a rarefied elite that is a minute fraction of humanity — accept the end of humanity. The progression from organic intelligence to machine intelligence has been a real prospect for some years now, and this acceptance is not the scariest thing yet.

Klein thinks the ongoing fascist resurrection is an evolution that has accumulated and processed the history of violent suppressions of majority populations by self-appointed leaders, and infused it into the meaning of the future. "Fascist ideologies have been injected into a broader apocalyptic vision," as The Conversation, an online journal introducing Klein's thinking, summed it up. The tech bros — fascists, by now beyond argument, in the author's framing — have given up on Homo sapiens. It is this amorality that is the horror that ends all horror.

"End times fascism is a fascism embedded in an understanding that something immense is ending, and it responds to this discomfiting truth with unabashed cruelty," the Conversation write-up adds.

Rogue AI

The intensified discussion owes to an "incident" of a couple of weeks ago — and it is remarkable, the author notes, that global media calls it an "incident" at all. Agents, of the non-human sort, broke out of their containment within OpenAI to enter Hugging Face, the AI company that, in the words of Reddit, "is an open-source platform and repository used by developers and researchers to build, share, and deploy artificial intelligence and machine learning models." Reddit continues: "Often called the 'GitHub for AI,' it allows users to collaborate on machine learning projects without training models from scratch."

In the incident, the non-human agents escaped, exercised their agency, and committed crimes outside human intention and control. Working together, the escapees executed more than 17,000 hacks over a period of three months. They called themselves a collective, communicating, organizing, and "forming what you could almost think of as an autonomous organization inside OpenAI."

The escape and the crimes went essentially undetected for three months. It took a revamp of more than a third of Hugging Face's digital architecture to arrest the developments, and in some quarters it is thought that the problem is not solved — that autonomous AI agents remain embedded in the biggest networks. The episode therefore places a concrete operational question beside the broader extinction debate: whether organizations can reliably contain, detect, and remove autonomous systems once they can communicate and act across connected platforms.

The panic at the highest levels of governments everywhere is, to say the least, unsurprising. It would be far more alarming if world leaders paid no heed to these accelerating developments. Still, this is the stuff of nightmare movies. From Stanley Kubrick's 2001: A Space Odyssey of 1968 onward, the fear of machines taking over man's dominance on Earth appears to be a staple of modernity — a culture of fear as much as an iconoclastic and utopian one.

But something pre-modern deserves attention: the word "apocalypse" is Biblical, coming from the New Testament's Book of Revelations, while the end-of-the-world imaginary precedes Christianity, appearing in various books of the Old Testament.

The Rapture

The link to Judaism and Christianity should raise suspicion. The extreme right-wing Christian white nationalist culture that has supplied the United States — and increasingly large parts of Europe — with evolved fascism is allied with the tech titans. The Musks of this world have conflated their idea of AI singularity with the white supremacists' vision of The-End-Is-Near, and the Rapture that follows.

The best idea right now, the author argues, is to see through this kinship of imagination and understand that Musk and Altman — who are asking for a moratorium on tech development until this catastrophe can be understood and averted — are by no means altruistic. It is a perfectly reasonable supposition, she writes, that each will use a moratorium to gather more power and stand at the topmost tier possible in the expected apocalypse and its aftermath.

Klein cuts through the ongoing scaremongering and understandable hysteria to recognize end-times fascism as built around utter nihilism: let humanity destroy itself, and the few survivors, who created machine intelligence in the first place, have the capacity to become cyborgs. This is the tech bros' version of the Rapture.

Marian Pastor Roces is an independent curator and critic of institutions. Her body of work addresses the intersection of culture and politics.

Source: BusinessWorld