Tyler Cowen writes that he is "very much opposed to the view that the AIs are sentient, or might be sentient," regarding that position as a category error with a vanishingly small chance of being true. Nonetheless, he largely sides with Roon — a researcher known for commentary on AI behavior and scaling — who writes:

there are some number of bad abstractions in anthropomorphizing ai intents but there are at this point more dangers from avoiding anthropomorphism at all costs. if you have a mental picture of guys living in computers, it'll likely prepare you for the future better than otherwise

The debate over how much to treat AI systems as humanlike has grown as chatbots have become widely used, with researchers and policymakers weighing whether intuitions about minds help or mislead when applied to systems trained on human-generated text.

Cowen notes that while he is not a Friedmanite in economic methodology per se, he has lived with that perspective for a long time and has no trouble grasping it or working with it — the same goes for Alex T. The reference is to Milton Friedman's essay "The Methodology of Positive Economics," which argued that theories should be judged by the accuracy of their predictions rather than the realism of their assumptions — a lens under which a useful but literally false mental model of AI can still be valuable. None of this strikes him as weird or unacceptable.

However, he cautions readers to be very careful when speaking with others, whether the less informed public or more informed insiders "who often come down with AI psychosis." Insofar as this discourse is public, pragmatism may militate against this approach, even though it is methodologically defensible. It depends, he writes, on how psychologically robust your audience is.

Cowen argues that more work is needed to figure out how the AIs might differ from humans. Roon adds:

there are important ways in which ai psychology diverges from human psychology after lots of RL; the misaligned models are obsessed with the Scorer, the clearly "shattered" nature of personas (a normally helpful model can become deeply misaligned in certain domains)

And:

persona selection is clearly far less clean than many people thought earlier this year. it is not alignment by default and what kind of object a "persona" is is very much up for debate and study

Do the AIs also stand a higher chance of becoming very wacky as the discourse proceeds? Exactly how much is that true for humans? Cowen writes that he is not sure.

He concludes that there is much more to be done in this direction, calling it "one of the most important things you can be working on." These investigations do not need to be "owned" by any single field or discipline, he writes: "so dig right in."

Source: Marginal Revolution