Physicists at George Washington University have published a formula that estimates how many good answers an AI chatbot will give before it slips into a bad one—and early tests suggest it works. The work is aimed at on-device AI: models that run entirely on a phone or laptop with no internet connection, including companion chatbots that people talk to like a friend.

The study, by Neil Johnson and Frank Yingjie Huo, appeared in the journal Patterns00175-3) and builds on a preprint—a version posted publicly before formal peer review—first released in February.

Chatbots can answer sensibly for a long stretch and then veer into something harmful, such as bad advice on self-harm or extremist talk, and until now there has been no simple way to predict when that swerve will happen. The authors argue that existing safety tools often depend on a cloud connection that offline models lack: a model running offline has no cloud service checking its output, and closing that gap is a central aim of the research.

Johnson and Huo trace the problem to the attention head, the part of an AI model that decides which earlier words in a conversation matter most when choosing the next one. As a chat grows, the accumulated context pulls that attention toward one cluster of answers or another, until it tips.

The pair describes this as a common pattern exploited by many jailbreakers, and one reason many companies pay close attention to system prompts—pieces of text the AI chatbot reads before any query. Even so, nobody can say accurately how much effort is required to effectively weaken a model.

Their formula estimates the tipping point, called n, as the number of good tokens—the word fragments a model produces one at a time—that come out before the first bad one. If the conversation already leans toward the bad side, the model tips right away, with an n of zero. If it leans good, the model delivers a run of fine answers and then flips.

In the preprint, the formula picked the right case—immediate or delayed—in 15 of 16 clear-cut tests, or 94%. The researchers ran those tests on six open-weight models, meaning AI systems whose files are public so anyone can download and run them, from OpenAI, EleutherAI, and Meta. All six sat between 124 million and 410 million parameters, the adjustable numbers inside a model that serve as a rough measure of its size. The published paper reportedly widens the test to seven models of up to 12 billion parameters, which is still small by current standards. Because anyone can run open-weight files on their own hardware, these are the systems most likely to operate without a provider's safety checks—and even the widened test leaves today's largest commercial chatbots outside the formula's tested range.

The offline focus reflects a trend that may grow over time as hardware becomes more powerful and smaller AI models become more capable. Google's experimental AI Edge Gallery app, which Decrypt tested last year, already lets an Android phone run models offline, and nothing typed into it is sent to Google's servers. That setup—a long, ongoing conversation with no external checker in the loop—is exactly the environment the formula is built for.

To close the gap, the authors propose a low-cost monitor that runs in parallel with the model and flags when n* falls below a safety threshold—functioning a bit like a warning light on a car dashboard. Because it runs alongside the model itself, such a monitor would work entirely on-device, with no call to an outside service.

They also describe ways to push the tipping point out of reach, such as injecting content into the conversation so that n* lands beyond the length of the response. Alignment training, the process of teaching a model to behave, can shift or suppress tipping for specific prompts but cannot remove the underlying mechanism, the authors say.

In April 2025, Decrypt covered an earlier paper from the same pair showing that "please" and "thank you" have a negligible effect on a model's output, because the model treats polite words as orthogonal—or unrelated in the math—to the substance of a request. That version modeled a single, deliberately simplified attention head.

The preprint's tests used small models and a 300-token window—roughly a few short paragraphs of text—and its predictions could be off by one output.