Claude misused (5/7) : A filter cannot tell a vaccine lab from a weapons lab

The same experiment that helps you build a vaccine helps you build the thing the vaccine is for. Biology has always worked that way, and it’s why the biological section of Anthropic’s September threat report reads so differently from the other six.

This is part five of seven, and part four was weapons. There, the intent was obvious, because nobody writes guidance code for a rocket by accident. Here Anthropic doesn’t name the institutions or the countries, and it says of the people involved, “We do not assert that they intended harm.” They’re working scientists in places where Claude isn’t offered, getting to it through VPNs and resellers.

Jacob Klein, Anthropic’s head of threat intelligence, described the problem to The New York Times, as Engadget reported. “You are not seeing someone in a comic book kind of way say, ‘Hey, I want to build a biological weapon to kill everybody,’” he said. “It’s an incredibly nuanced situation.”

A filter reads the request. It has no way to read the person making it.

The first is a platform built to route around refusals. In May 2026, Claude’s biology classifier blocked a request for help writing a grant application. The work was on the chikungunya virus, aimed at making it spread better and dodge immunity, and it was to be carried out at a military research institute. Anthropic looked further and found the request came through a platform serving dozens of life-science researchers in countries it doesn’t supply, with the traffic tunnelled through US infrastructure. The platform’s developers had built a fallback so that anything Claude refused was sent to a competitor’s model. After the ban they were back within days. They then added a pre-launch test that failed if a rule-breaking biology prompt reached Claude at all, and Claude wrote much of that code, having been told it was to reduce over-refusal.

The second is the filter doing its job. A researcher spent weeks, and thousands of messages, planning bird flu experiments about adapting the virus to mammals, which is the direction you’d least want it to go. The classifiers on Anthropic’s stronger models block that work, so every exchange happened on its weakest ones. Anthropic’s estimate is that the help came to mostly clerical assistance with data analysis and study design. I’d call that a win. The report doesn’t stop there, though. It treats the case as evidence that such a programme exists, probably has physical access to the virus, and wants frontier AI badly enough to hide who it is.

The third is the one nothing was built to stop. A researcher on a national programme used Claude to redesign a set of toxins by computer, framed as therapeutic work, and co-wrote the quarterly progress reports with it. They told Claude to keep the descriptions of what the toxins actually were deliberately vague. The classifier didn’t step in, and Anthropic says that was by design, because it exists to stop novices recreating known weapons and has no way to judge a new compound that could become a medicine or a poison. In a related case, a reseller’s customer had Claude’s Opus 5 draft a complete grant application about weakening a virus from the smallpox family in about an hour, from hypothesis to statistics plan, and that wasn’t blocked either.

Some proportion helps here. Anthropic says it swept 30 days of activity tied to what it calls adversarial state institutions and found about 35 distinct research efforts, “most of them ordinary civilian science.” It also says it doesn’t read these cases as a sign that a biological threat helped along by Claude is imminent. What they show is that serious dual-use labs want these models and will route around a regional block to get them.

Anthropic’s own conclusion is that a classifier can’t carry this alone, and that “the only safe way to serve frontier biological capabilities is to offer them in trusted user programs.” In practice that means knowing which lab is asking and keeping a record of what it asked. Which lands on something awkward for the rest of us. That first platform hid its traffic behind zero data retention, the setting where the provider keeps no record of what you sent, and it’s the same setting every privacy-minded company asks for. I want that setting too. I also can’t argue with the point that you won’t spot a pattern across weeks of messages you never kept.

Most of us won’t ever go near a virus, so the useful lesson is about the phrase “the model has safeguards.” When a vendor says it, ask which requests the filter was designed to catch and what happens to everything else, because in this report the honest answer for whole categories was that nothing happens, on purpose. And expect ID checks and retained logs to become the price of the most capable science tools, so it’s worth deciding now what you’d trade for that.

The biological section is here, and it’s the most carefully hedged part of the report, which in this case is to its credit. Part six is the one fraud case in the report, a network of dating apps where most of the profiles were Claude.