Anthropic adds Claude 4 security measures to limit risk of users developing weapons

Anthropic adds Claude 4 security measures to limit risk of users developing weapons


Omar Marques | Lightrocket | Getty Images

Anthropic on Thursday said it activated a tighter artificial intelligence control for Claude Opus 4, its latest AI model.

The new AI Safety Level 3 (ASL-3) controls are to “limit the risk of Claude being misused specifically for the development or acquisition of chemical, biological, radiological, and nuclear (CBRN) weapons,” the company wrote in a blog post.

The company, which is backed by Amazon, said it was taking the measures as a precaution and that the team had not yet determined if Opus 4 has crossed the benchmark that would require that protection.

Anthropic announced Claude Opus 4 and Claude Sonnet 4 on Thursday, touting the advanced ability of the models to “analyze thousands of data sources, execute long-running tasks, write human-quality content, and perform complex actions,” per a release.

The company said Sonnet 4 did not need the tighter controls.

Jared Kaplan, Anthropic’s chief science officer, noted that the advanced nature of the new Claude models has its challenges.

“The more complex the task is, the more risk there is that the model is going to kind of go off the rails … and we’re really focused on addressing that so that people can really delegate a lot of work at once to our models,” he said.

The company released an updated safety policy in March addressing the risks involved with AI models and the ability to help users develop chemical and biological weapons.

Major safety questions remain about a technology that is advancing at a breakneck pace and has shown worrying cracks in safety and accuracy.

Last week, Elon Musk’s Grok chatbot from xAI continued to bring up the topic of “white genocide” in South Africa in responses to unrelated comments.

The company later attributed the bizarre behavior to an “unauthorized modification.”

Olivia Gambelin, AI ethicist and author of the book “Responsible AI,” said the Grok example shows how easily these models can be tampered with “at will.”

AI researchers and experts told CNBC that the push from the power players to prioritize profits over research has led to companies taking shortcuts and forgoing rigorous testing.

James White, chief technology officer at cybersecurity startup CalypsoAI, said companies sacrificing security for advancement means models are less likely to reject malicious prompts.

“The models are getting better, but they’re also more likely to be good at bad stuff,” said White, whose company performs safety and security audits of Meta, Google, OpenAI and other companies. “It’s easier to trick them to do bad stuff.”

CNBC’s Hayden Field and Jonathan Vanian contributed to this report.



Source

Tesla shares tumble after Trump says DOGE should look at Elon Musk’s subsidies
World

Tesla shares tumble after Trump says DOGE should look at Elon Musk’s subsidies

Elon Musk speaks with U.S. President-elect Donald Trump as they watch the launch of the sixth test flight of the SpaceX Starship rocket on November 19, 2024 in Brownsville, Texas. Brandon Bell | Getty Images News | Getty Images Shares of Tesla were under pressure on Tuesday after President Donald Trump said in a late […]

Read More
Euro rises after euro zone inflation hits ECB’s 2% target
World

Euro rises after euro zone inflation hits ECB’s 2% target

European government borrowing costs fall Yields on European government bonds moved lower in the wake of the latest euro zone inflation print, which showed inflation hit the ECB’s 2% target in June. The yield on the German 10-year bund — seen as a benchmark for the euro zone — was 4 basis points lower by […]

Read More
Bank of England chief sees downward interest rate trend as UK hunts for growth
World

Bank of England chief sees downward interest rate trend as UK hunts for growth

Andrew Bailey, governor of the Bank of England, at the central bank’s headquarters in the City of London, U.K., on Nov. 29, 2024.  Hollie Adams | Bloomberg | Getty Images Bank of England Governor Andrew Bailey told CNBC Tuesday that “the path of interest rates will continue to be gradually downwards,” as the central bank […]

Read More