California Governor Proposes Independent Watchdogs Within AI Safety System — Campus Technology

[ad_1]

California Governor Proposes Independent Watchdogs Within AI Safety System

California Governor Gavin Newsom is proposing a new model of artificial intelligence regulation that would give independent organizations a formal role in evaluating the safety claims made by leading AI companies.

The effort could eventually include placing outside evaluators inside frontier AI laboratories and requiring companies to maintain an independently tested emergency shutdown mechanism, commonly described as a “kill switch.”

Those requirements are not yet California law.

Newsom’s Sept. 18 executive order directs state officials to study the proposals and recommend possible changes to existing law by Nov. 16. It does not immediately require OpenAI, Anthropic, Google, or other developers to install a shutdown mechanism.

The distinction matters because the phrase “AI kill switch” suggests a technical safeguard California has already imposed. For now, it is one element of a broader oversight system the state is assembling around the developers of its most powerful AI models.

Under the order, the California Government Operations Agency and the Governor’s Office of Emergency Services must consult outside experts about whether the state should require large frontier developers to embed independent verification organizations in their laboratories.

The evaluators could conduct periodic audits and determine whether company safety frameworks and risk assessments meet adequate standards. They could also test whether an emergency shutdown mechanism would work as intended.

The order asks officials to consider expanding the definition of a reportable critical safety incident to include cases in which an AI system appears to evade controls or operate beyond its intended boundaries.

Newsom said the action was prompted by recent incidents involving AI agents defeating security restrictions, as well as the absence of comprehensive federal legislation.

“We’re not waiting to act,” Newsom said in the state’s announcement of the order. “We’re going to speed up our work on substantial and responsible AI oversight before it’s too late.”

The executive order builds on two laws Newsom signed on Sept. 9. Together, Senate Bill 813 and Assembly Bill 1405 begin regulating the organizations hired to inspect AI systems.

From Disclosure to Verification

California’s earlier frontier AI law, SB 53, requires large developers to publish safety frameworks and report specified critical incidents. The new laws begin addressing a separate question: Who checks whether those disclosures and assessments are reliable?

SB 813 requires the Government Operations Agency to develop standards for designating independent verification organizations, or IVOs. Those organizations must demonstrate that they have the expertise to assess AI risks and explain the measurements and methods supporting their conclusions.

[ad_2]

Source link


Comments

Leave a Reply

Your email address will not be published. Required fields are marked *