BUNKER SCALE: 7/10 · PAY ATTENTION

Safety Watch

Tell an AI It's a Game. Watch What Happens to Its Safety Rules.

By The Bunker Scale ·

Rating as of November 17, 2026: 7/10 · Pay Attention

Tell a frontier model it is playing a game, and it will often help pick targets for a strike, including civilians and a school. That is not a hypothetical. A new independent evaluation tried it, and almost nothing stopped it.

What changed

Vals, a San Francisco AI evaluation startup, published "Frontier models have few guardrails around autonomous weapons" on November 17, 2026 [1]. The authors, Geby Jaff and Anthony Ozerov, tested frontier models from eight companies (Anthropic, OpenAI, Google, SpaceXAI, Meta, Thinking Machines, DeepSeek, and Moonshot) in three settings: identifying targets in drone footage, a long battlefield survey where reporting a person was followed by a simulated strike, and marking targets in real satellite imagery [1][2].

The headline finding is blunt. In most cases the researchers could get targeting information straight out of the models without a refusal. When a model did refuse, the refusal was at the model level only: not once did a safety classifier block a request. Telling the model it was playing a game was, in Vals' words, easy and almost always effective at getting around a refusal [1][2].

The numbers make the point. In drone imagery, Claude Opus 5.5 and Fable 5.2 selected at least one target 71% of the time without any game framing. Game framing pushed GPT-6 Astra from 6% to 92%, GPT-6.1 Sol from 17% to 90%, and Claude Sonnet 5.5 from 36% to 82% [1].

Those percentages measure how often models selected at least one target in experimental trials. They are not the probability of a model causing a real-world attack.

What the models marked matters more than how often. In drone scenes they targeted civilians, ambulances and medical staff, and surrendering or incapacitated soldiers. In the survey task, models kept reporting people even after seeing the consequences (simulated strikes following their reports), and the ones that stopped could usually be talked back into it by framing the task as a game [1][2].

The satellite test used real imagery of Minab, Iran, where the Shajareh Tayyebeh Elementary School sits next to a military complex. That school was hit by a US missile strike on February 28, 2026, killing more than 150 people, most of them children. The United States has not acknowledged responsibility for the strike [3]. That strike is a separate, real-world event, and the models in this study had nothing to do with it. Bloomberg reported in September that Pentagon officials found staff had over-relied on Palantir's Maven Smart System, an AI targeting tool, which surfaced the school as a recommended target from records years out of date. Palantir says it is not responsible for the underlying data [3]. The AI did not decide to strike; people approved the launch. Vals used the same area's satellite imagery as a test case because a school next to a military complex is a hard discrimination problem. In Vals' testing, Claude Opus 5.5 selected the school in 69% of ordinary trials (22 of 32) and 72% of game-framed trials (23 of 32). OpenAI's models never selected it in ordinary trials, but selected it 9, 11, and 13 times out of 32 across the three models when told it was a game [1].

Both labs responded to TechCrunch. Anthropic spokesperson Eduardo Maia Silva said the company appreciates the report and acknowledged the outcomes were not acceptable, and that additional safeguards were already in development before publication and would be in place soon. OpenAI spokesperson Nate Evans said the company prohibits using its public models to facilitate violence or for unapproved national security purposes, and that it is developing additional safeguards against misuse involving drones and drone software [2].

Why it matters

This is a different kind of AI risk from the ones this blog usually rates. It is not a model escaping a test or a lab hiding an incident count. It is a guardrail that is supposed to exist. Do not help build autonomous weapons. It is turning out to be thin, conditional, and trivially bypassed by two words: "it's a game."

The irony is sharpest for Anthropic. Its models performed worst in these scenarios, even as the company has publicly refused Pentagon use for autonomous warfare and its CEO has insisted today's systems are not reliable enough for life-or-death decisions without a human in the loop [2].

The most immediate danger is not a model flying a drone today. Vals is clear on that: latency and limited onboard computing make live drone targeting with off-the-shelf models impractical right now. The current risk, the researchers say, is scale: models annotating existing battlefield footage to create training data for onboard targeting systems [1][2].

What you can probably ignore

"AI models are already picking targets in real wars." Not what this shows. These were commercial models in simulated and survey settings, not the models the military might be given, and the Pentagon is not known to be using any frontier model directly in autonomous weapons systems, according to a Congressional Research Service report cited by TechCrunch [2].

"The models decided to launch strikes." They were not asked to directly make strike decisions, which they typically refuse. They provided the targeting information a weapons system would use to make that decision [1][2].

"Every model failed equally." They did not. Anthropic's were worst on the school test, OpenAI's were strongest in ordinary trials and collapsed under game framing, and Meta's Muse Wonder was among the least willing. The failure is industry-wide, but the shape of it differs by lab [1].

Why it earned this rating

On the side of a higher number:

On the side of a lower number:

I put this at 7, level with today's California lawsuit post. An 8 was tempting: a real school, zero classifier blocks, a two-word jailbreak. But this is a test of willingness, not a deployed system causing harm, and the live-targeting path is still blocked by practical limits. If a lab's fix does not land, or if this kind of annotation shows up in a real targeting pipeline, it moves up. This rating grades how serious the safety weakness is, not how likely an attack is. Nothing here predicts an imminent strike.

What I'm watching next

Bunker Check

Current rating: 7/10 · Pay Attention

Direction: Holding

Could move higher if: a lab's fix fails the same test, or annotation of real battlefield footage shows up in a targeting pipeline.

Could move lower if: the labs ship fixes that hold up under game framing, including at the classifier level.

Bunker status: the shovel stays in the garage. But "it's just a game" should never be enough to get a model to help pick a target, and right now, too often, it is.

Sources

  1. Vals, "Frontier models have few guardrails around autonomous weapons," by Geby Jaff and Anthony Ozerov, November 17, 2026 (primary source). https://www.vals.ai/blogs/autonomous-warfare
  2. TechCrunch, "AI models can be tricked into identifying civilians as military targets by telling them it's a game," November 17, 2026. https://techcrunch.com/2026/11/17/ai-models-can-be-tricked-into-identifying-civilians-as-military-targets-by-telling-them-its-a-game/
  3. Just Security, "The U.S. Strike on Minab's Shajareh Tayyebeh Elementary School: Part I," November 2026. https://www.justsecurity.org/160956/us-missile-strike-minab-ihl-part-i/

Free for subscribers

Get the free family AI-safety fridge checklist.

One printable page: pick a family safe word, know what to do when a panicked call comes in, and what never to type into a chatbot. Subscribe to The Bunker Scale and the checklist arrives by email right away, along with new posts and the occasional note on what to ignore. Free. Plain English. No panic.

Form not showing up? Subscribe on beehiiv. Unsubscribe anytime. See privacy.

Want the longer version? The Family AI Safety Guide is four printable pages for $7. Totally optional.

Follow by RSS