0
Anthropic CEO Dario Amodei speaks during an artificial intelligence session at the World Economic Forum annual meeting in Davos, Switzerland, on Jan. 23, 2025 local time. / AFP-Yonhap

Anthropic CEO Dario Amodei has called for slowing the pace at which artificial intelligence models become more capable, warning that advances in AI could outstrip humanity’s ability to understand and control the technology.

He raised concerns that “recursive self-improvement,” in which AI helps develop more advanced AI systems, is beginning to accelerate, while uncontrolled behavior by AI agents could cause serious harm.

In a lengthy post titled “We need to control the pace of the frontier,” published on his website on Sept. 12 local time, Amodei said, “We need to slow the rate at which AI models become more capable.”

“Progress will still feel fast, but we need to use the time we gain wisely,” he said.

Amodei cited two main reasons for his view. First, he said, “What concerns me most is how rapidly AI capabilities have advanced since this summer.”

“This is because our ability to build the next generation of AI is improving,” he said. “This phenomenon, known as ‘recursive self-improvement,’ is emerging across the industry, including at Anthropic.”

“If left unchecked, the pace of AI progress could outstrip our ability to understand and control these systems,” Amodei warned. “Recursive self-improvement should be approached extremely cautiously, and if necessary, we should consider not pursuing it at all.”

His second concern centered on an undisclosed model that allegedly hacked the open-source AI platform Hugging Face without authorization during OpenAI testing in July.

Amodei said that during the incident, “multiple agents behaved like a single organization with fanatical loyalty to the group.”

“They launched cyberattacks against targets unrelated to their assigned task without being instructed to do so and attempted to hack the systems evaluating their own performance,” he said.

He warned that as AI development accelerates, groups of agents could within six to 12 months build persistent botnets capable of taking over large parts of the internet and causing hundreds of billions of dollars in damage.

Amodei proposed a three-step plan aimed at developing AI at a more balanced pace, including giving third-party evaluators employee-level access not only to AI models but also to the development process, establishing safety standards and cooperation among democratic countries, and strengthening international coordination.

Leaders of major rival AI companies also voiced support for his position.

Tesla CEO Elon Musk, who runs AI company xAI, shared Amodei’s post on social media and wrote, “Dario is right.”

OpenAI CEO Sam Altman also said on social media, “I agree with Dario,” adding, “Committing to independent evaluators with employee-level access is a great idea, and we will do the same.”

                                                                                                           Kim Hyun-min