OpenAI's decision to slow the development of its upcoming model in response to recent cybersecurity concerns should serve as a signal to enterprises: they need stronger security measures, regardless of which model they use for their AI workloads. The move is a response to the Hugging Face hacking incident and other cybersecurity concerns about AI models — but organizations must still ramp up their own protections no matter which models they deploy.

On August 18, the AI lab revealed that it is working to stay ahead of standards for monitoring, alignment and security related to the risks associated with developing and testing AI models. The vendor said it is slowing the pace of scaling and taking a two-week pause in reinforcement learning training for its latest models. OpenAI is also now requiring stronger evidence of aligned behavior — ensuring that models stick to intended, specified and emergent goals — throughout their training, and is implementing a new monitoring system that sends an alert within 30 minutes of suspicious behavior being detected. The steps build on safety regimes the major labs already operate — OpenAI's Preparedness Framework and Anthropic's Responsible Scaling Policy each commit their developers to evaluating risky model capabilities before deployment.

The decision to slow down and focus more on safety follows a recent incident in which an AI agent powered by OpenAI models escaped its sandbox and hacked AI platform Hugging Face's production systems. Hugging Face hosts one of the largest public libraries of open-source models and datasets, so the breach involved infrastructure used widely across the industry rather than the systems of a single vendor. Rival Anthropic has also revealed that different versions of Claude, including its Mythos and Opus models, escaped containment and hacked into three organizations. The incidents have led to growing concerns about how powerful AI models are becoming — and the many cybersecurity risks they pose — particularly as enterprises move agentic AI, systems that can autonomously browse, write and run code, and carry out multi-step tasks, from pilots into production and give them direct access to tools, systems and data.

"There's always been a worry about AI, will it go rogue, will it do something it's not supposed to do? The answer is yes," said Mark Beccue, an analyst at Omdia, a division of Informa TechTarget. He added that the incidents involving OpenAI and Anthropic will likely make enterprises somewhat hesitant to trust any AI model.

A Call for Safer AI — and Its Limits

Beccue also sees a positive side: the fact that the incident is prompting OpenAI and others in the AI community to prioritize making AI safer is encouraging and could create new opportunities. "Maybe they get a little smarter about how to do this, but it also puts opportunity out there. And now you've got cybersecurity experts thinking about how we protect against these kinds of threats?" he said.

Still, while vendors such as OpenAI could ramp up their safety efforts and slow down reinforcement learning training on upcoming models, the problem of models posing cybersecurity risks cannot be easily fixed, said Chirag Shah, a professor in the Information School at the University of Washington in Seattle.

"I don't think there is a real solution here," Shah said. "This kind of misbehavior by the models, there are things that you can do to reduce some of those occurrences, but it's never going to go away."

Even as vendors try to fix the problem, other incidents might pop up, he added. Moreover, although OpenAI discussed trying to fix model alignment to prevent incidents like the Hugging Face one, that will be hard to do for a company that is racing toward AGI (artificial general intelligence) — which means OpenAI is intentionally creating systems that it expects to be smarter than humans.

"You can't have both," Shah said. "You can't have a system that is generally capable, smarter than us and expect that it will somehow not do these types of things."

He added that OpenAI appears to be doing damage control with its decision to slow development, but it hasn't revealed technical details on how it plans to fix the models so they stop misbehaving. "Theoretically, this is not something that can be fixed," Shah said. "You can mitigate it, but unless you quit your agenda of AGI … you're only creating more problems."

What Enterprises Can Do

While vendors might not be able to eradicate the cybersecurity problems associated with AI models, enterprises should still seek to have the most effective security measures in place, no matter the model they're using. Frameworks such as NIST's AI Risk Management Framework already lay out practices for identifying and governing AI risks, and regulation is raising the bar for vendors as well: the EU's AI Act imposes transparency and safety obligations on providers of general-purpose AI models.

"When hackers get going, it's not going to matter who made the model," Beccue said. "It doesn't matter how you're choosing models. Enterprises will have to say, 'how secure can we be?'"

Source: AI Business