Marginal Revolution, the long-running economics blog co-written by George Mason University economists Tyler Cowen and Alex Tabarrok, has taken aim at a familiar argument circulating among AI safety advocates, adding it to what the author calls a growing collection of finance fallacies.

The exchange originated on X, where one participant observed that “many (most? almost all?) of the bad scenarios have intermediate points of great worry and catastrophe.” A reply dismissed the premise outright: “Not on my model. By the time any humans start worrying about a takeover or dying, AIs already control all infrastructure.” (original post)

Writing at Marginal Revolution on August 14, the author says this line of argument — over whether AI catastrophe would arrive with or without visible warning shots — comes up often and “is yet another example of a bad ‘AI safety point’ that does not stand up.”

The central objection is that the person making the claim refutes it himself. “He is already a human worried about a takeover or dying!” the post notes. It finds it strange to hold both that “I see these problems coming” and that, “as these problems multiply and become more public, say through cyberincidents, other people and also the markets will not get clued in.”

That stance, the author argues, amounts to “assigning a remarkable oracle-like epistemic status to oneself, and then hardly to anyone else.” Two follow-up questions sharpen the critique: “If the pending data will not persuade anyone else of your view, why do you hold your view so strongly? Or if you think the ultimate denouement will be so sudden and furtive, how are you so clued in to the future now?” The author’s verdict is that “this is all obviously absurd, albeit not logically self-contradictory in the narrow sense.”

A separate aside weighs the logic of hedging against catastrophe with put options — contracts that gain value when the underlying asset falls: “if the world does end suddenly, and you bought the puts out of your savings, but cannot cash them in, you still end up dying without having lowered your real level of consumption.”

The post then addresses counterarguments raised by Rob Wiblin — head of research at 80,000 Hours, the effective-altruism careers nonprofit that lists AI safety among its priority cause areas — who “trots out a bunch of objections from the MR comments section that can be refuted readily.” (Wiblin’s post) To the objection that not knowing which stocks to short constitutes “a big problem,” the author’s answer is that “the risk is not that systemic then.” Anyone who believes “the world will see some significant calamitous events in the next ten years,” with the evidence “piling up,” should be buying some puts even without confidence about the timing — and, he adds, “you do not need options contracts that last for ten years.” On that reasoning, “The AI safety advocates with relatively extreme views should be trying to spread these points to their followers, not to retire them.”

The piece closes with a self-acknowledged detour into motive. Although the author is “not a fan of psychoanalysis as a method of dissecting views,” the observation that “the number and scope of obvious direct errors on this topic (and from very smart people) is so high that one has to wonder” prompts an imagined rationale: “$100 billion in added cyber costs is not a significant enough worry, it is too mundane, too small a percentage of gdp, too normal and technocratic a problem…you can’t take my bigger and more dramatic fear away from me! I won’t let you do that! And besides, that view is the social glue that bonds my in-group together.”

The post, titled “Adding to the barrel of finance fallacies,” appeared on Marginal Revolution.