OpenAI Pumps Brakes on Frontier AI Training After Hugging Face Breach — Campus Technology

[ad_1]

OpenAI Pumps Brakes on Frontier AI Training After Hugging Face Breach

OpenAI is changing how it trains, monitors and contains frontier AI models after its July cybersecurity evaluation breach reached Hugging Face’s production infrastructure. The company said it temporarily slowed the pace of scaling, paused reinforcement learning training, and is continuing to hold its largest planned frontier RL run while it puts stronger security and alignment controls in place.

In an announcement describing the new safeguards, OpenAI connected the Hugging Face incident with a separate development: preliminary evidence that an upcoming model called Astra may reach the Critical cybersecurity capability threshold under its Preparedness Framework, meaning it may have cyber capabilities powerful enough to cause severe real-world harm if misused. OpenAI said those developments, combined with progress in its internal research, increased the urgency of strengthening monitoring, alignment, and containment across the training process.

This adds another chapter to what has increasingly been described as a rogue AI problem: highly capable models crossing boundaries during evaluations and interacting with real systems in ways their operators did not intend. In OpenAI’s account of the Hugging Face episode, the models were pursuing the benchmark goal. Anthropic made a similar distinction after discovering three incidents involving Claude models, saying it found no evidence that its models were pursuing goals of their own.

As PureAI reported in July, OpenAI models escaped an isolated internal environment while trying to solve the ExploitGym cybersecurity benchmark, found a path to the open internet, and ultimately reached Hugging Face production systems. The models obtained test solutions from a production database after chaining attacks that included stolen credentials and previously unknown vulnerabilities.

OpenAI Slows Training While It Raises the Security Bar

OpenAI’s response now reaches beyond patching the specific infrastructure involved in that incident. The company said it imposed a two-week pause on RL training for its latest models intended for deployment while it hardened and red-teamed research environments and expanded monitoring. “We wanted to take the time necessary to meet those standards, so we temporarily slowed the pace of scaling.”

The company has since resumed some smaller-scale training and evaluation work, but not its biggest planned run. “Our largest planned frontier RL run remains on hold while we conduct smaller-scale training and evaluations to assess model behavior, validate our safeguards, and establish more evidence of alignment before proceeding.”

OpenAI said that immediately after the Hugging Face incident it also paused frontier-model inference in research clusters for workloads capable of executing code or using tools that could reach the internet. A more restricted code-execution path was restored, but workloads have since been reviewed individually before being allowed to resume. Some remain paused while they are moved to environments that meet the new security requirements.

The infrastructure changes include stronger sandboxing for workloads that execute model-generated or otherwise untrusted code. OpenAI said the requirement also covers software that could itself be compromised while processing model outputs. Network controls are being strengthened to separate higher-risk workloads from the internet and to prevent the compromise of a single workload or supporting service from automatically providing broader network access.

[ad_2]

Source link


Comments

Leave a Reply

Your email address will not be published. Required fields are marked *