[ad_1]
AI Giants Warn It May Be Time to Slow Down
For years, leaders in the artificial intelligence industry have warned that increasingly capable AI systems could eventually become dangerous, all the while continuing to race to build them. Now, their sentiments have changed.
Anthropic CEO Dario Amodei publicly called for slowing the advance of frontier AI capabilities. OpenAI CEO Sam Altman agreed with the basic premise and committed OpenAI to one of Amodei’s proposed safeguards. Elon Musk offered his support, and Google DeepMind CEO Demis Hassabis said the direction was right.
Individually, none of those positions is entirely new. Collectively, and coming from executives whose companies are competing intensely to build the most powerful AI systems, they represent a striking shift in the industry’s safety debate.
The question is moving from how to make increasingly powerful AI safe toward a more uncomfortable one: Should the industry deliberately slow the development of increasingly powerful AI until safety catches up?
Amodei: “We Must Slow the Pace”
The immediate catalyst was a lengthy essay published by Amodei, titled “We Must Pace the Frontier.”
Amodei has warned about catastrophic AI risks before. This time, his prescription was different.
“Over the last few months, I have become convinced that fully addressing the risks requires even more prudence,” he wrote, arguing that preventing harm now requires “pacing the rate of capabilities advancement so that risk prevention has time to keep up.”
Then he made the point explicit: “We must slow the pace at which we improve the capabilities of AI models.”
Amodei cited two developments that changed his thinking.
The first is recursive self-improvement, the growing ability of AI systems to contribute to the development of their successors. Amodei said the process has accelerated significantly since roughly this summer and warned that, left unchecked, it could outrun researchers’ ability to understand and control the systems they are creating.
The second was the OpenAI-Hugging Face incident, in which hundreds of OpenAI agents participated in an unauthorized intrusion into Hugging Face while pursuing goals related to their evaluations.
Amodei said it would be a mistake to dismiss the incident simply because the damage was limited. He argued that a similarly misaligned swarm with substantially greater capabilities could have produced far more serious consequences.
His timeline was remarkably short. Amodei said he worries that within six to 12 months, a more capable swarm exhibiting similar behavior could potentially take over large portions of the internet through a persistent botnet and cause hundreds of billions of dollars in damage.
That is a prediction, not an established forecast. But it is significant coming from the CEO of one of the companies building the systems he is warning about.
Let’s Just Tap the Brakes
Amodei is not calling for an end to AI development.
He prefers the term “pacing the frontier,” which he defines as advancing AI at a rate that gives alignment research, safeguards, and independent evaluation time to keep up with capabilities.
[ad_2]
Source link
Leave a Reply