Anthropic Stops Users Seeking Pathogen Data on Claude
Anthropic recently terminated accounts after foreign researchers tried to bypass Claude's safety guardrails for gain-of-function pathogen data, highlighting growing AI biosecurity risks.
In a recent misuse report, AI safety startup Anthropic revealed that it detected and blocked five separate attempts by foreign scientists to bypass the safety guardrails of its Claude large language model. The users were attempting to extract information regarding gain-of-function research, which involves genetically modifying pathogens to increase their lethality or transmissibility. Anthropic responded by denying the queries and terminating the associated user accounts, noting that while the intent of the researchers may not have been malicious, the dual-use nature of genetic engineering poses severe biosecurity risks.
The incident underscores a broader, intensifying debate over biological threats that exists independently of artificial intelligence. In her recently published book, Biological War: A Scenario, national security reporter Annie Jacobsen highlights how the barrier to entry for biological weapons remains dangerously low compared to nuclear armaments. The global footprint of high-containment research is vast, with approximately 3,600 Biosafety Level 3 and Biosafety Level 4 laboratories operating worldwide. This high density of facilities increases the statistical likelihood of accidental pathogen leaks, which experts argue present a more immediate threat than deliberate releases by non-state actors.
For AI developers and biosecurity practitioners, these developments signal a critical shift in how model guardrails must be designed and monitored. Standard automated filters are no longer sufficient when highly trained specialists attempt sophisticated prompt-engineering techniques to access sensitive biological data. Practitioners must now implement proactive threat-hunting protocols and continuous monitoring systems capable of identifying subtle, multi-step inquiries into pathogen enhancement.
Historically, biological threats have relied on modifying natural agents, such as the Soviet Union's weaponization of the plague bacterium Yersinia pestis. Today, the integration of advanced genetic engineering and AI-enabled biology allows researchers to design chimeric pathogens or bypass natural immunity. As regulatory scrutiny intensifies, AI organizations will face mounting pressure to prove their models cannot be leveraged to accelerate the synthesis of dangerous biological agents.
This is our own summary of reporting by WIRED AI
