A major technology company says bad actors recently tried to use their AI tool to create a biological weapon, as well as other very dangerous activities.
Anthropic reports it blocked attempts to use its artificial intelligence models for malicious cyberattacks, surveillance, and research that could have led to weapons of mass destruction.
The company’s report came two days after one of its researchers resigned, warning that the people building AI earnestly believe that it could kill us all by the end of the decade.
* Another AI Jailbreak: Anthropic’s Claude Escapes a Test and Hacks Outside Groups
Anthropic says it has added stronger safeguards in its latest models to restrict biological research that could also be used to make weapons.
“We’re publishing this work because we believe we have a responsibility to disclose malicious misuse of our services. As models become increasingly capable, their risks will increase, unless AI developers and society’s defenders act to make them safer,” the company said.
One instance involved gain-of-function research, similar to the now-dominant theory about the origin of the COVID-19 virus.
“The work discussed in the application involved gain-of-function research (that is, research that genetically alters an organism to create a new or enhanced biological property) on the chikungunya virus. This gain-of-function research was aimed at the virus’ transmissibility and immune evasion properties,” the report said.
Chikungunya is a mosquito-borne virus that causes debilitating symptoms such as severe pain and fever. The request involved a grant proposal for research seeking to enhance mutations to make the virus progressively more harmful.
Anthropic admitted that its latest, most powerful versions of AI “are capable of assisting in a range of complex scientific research tasks,” and the company “cannot make that same assurance” that they couldn’t be used to create a bio-weapon. Therefore, the tech company has applied “stronger safeguards that restrict access to a wide range of dual-use biological research queries” in its more recent models, such as Claude Fable 5.
* ‘Unprecedented’: AI Agent Escapes Secured Testing Environment and Hacks Another AI
As AI models grow more powerful, elaborate cyberattacks no longer require sophisticated skills, and even lone individuals can create threats that would not have been possible even a year ago, Anthropic said.
The company also found groups that created hundreds of fake social media accounts that look like they belong to ordinary people and then posted material amplifying the same political view over the course of a week. The company outlined nine such cases it found, originating in Russia, Iran, Turkey, spots around the Persian Gulf, South Asia, Africa and Europe.
Congress is looking at getting involved with new proposals to regulate AI as concerns escalate inside and outside of the industry about the technology’s growing potential to escape human control and take unsanctioned actions.
