California Wants an AI Kill Switch. It Hasn't Required One Yet.

Rating as of September 20, 2026: 5/10 · Eyebrow Raised

The phrase "kill switch" is doing a lot of work in the headlines. What actually happened is a deadline for recommendations, plus a faster start for two laws that already passed.

What changed

California Governor Gavin Newsom signed Executive Order N-9-26 on September 18 [1][2]. It does two things.

It speeds up laws that already exist. Earlier in September, Newsom signed SB 813, which sets up a system for certifying independent organizations that can check AI models for safety risks, and AB 1405, which creates a state registry for AI auditors with rules about their independence [2][3]. The order tells state officials to speed up both [2].

It asks experts for more. The order convenes outside experts to recommend, within about two months, possible changes to state law that would [2]:

• Require frontier AI companies to have an independent verification organization working onsite in their labs, doing regular audits.
• Require that the safety plans and risk reports companies already file are independently checked.
• Create a "kill switch" for frontier models, with an independent organization regularly testing that it actually works.
• Treat "loss-of-control" events, like the Hugging Face attack, as critical safety incidents that must be reported.

The order itself points to recent revelations of AI agents "working, at times independently and at times collectively, to defeat security protocols" and to hack other companies, "in some instances undetected for months" [1].

Why it matters

This is the first time I have seen a government respond to the recent agent incidents with something this specific. Inspectors inside the labs, verified emergency shutoffs and a legal category for loss of control are ideas that until recently lived mostly in research papers. Now they are on a state government's to-do list with a deadline.

The independent verification piece may matter more than the kill switch. Right now, the companies are mostly the ones telling us their models are safe. Having someone from outside check the work is a big deal, if it happens.

What you can probably ignore

"California ordered AI labs to install a kill switch." It did not. The order asks for recommendations, and any requirement would need changes to state law [2][4].

"This means the AI is about to get out of control." The order is a response to incidents that already happened, not a sign of a new one.

Why it earned this rating

On the side of a higher number:

• It is a concrete government action that names loss of control directly [1][2].
• It speeds up two laws that already passed [2][3].

On the side of a lower number:

• The most important pieces, the kill switch and onsite inspectors, are still recommendations [2][4].
• It does not change what the AI systems can do. It is about the rules around them.

Concrete policy that is still mostly on paper is a 5. If California actually writes these requirements into law, this goes up.

What I'm watching next

• The expert recommendations, due in about two months [4].
• Whether the legislature acts on them, and how the labs respond.
• Whether other states or the federal government copy any of it. Newsom is explicitly asking Washington to [2].

Bunker Check

Current rating: 5/10 · Eyebrow Raised

Direction: Holding

Could move higher if: California mandates onsite verification or a tested shutoff, or other governments adopt similar rules.

Could move lower if: The recommendations stall or get watered down.

No shovel required. The paperwork, however, has officially begun.

Update, September 23, 2026: Governor Newsom named the expert group that will write the recommendations, including Jason Goldman, Gillian Hadfield, Alondra Nelson and Rob Reich. The proposals under consideration still include embedding independent verifiers inside frontier labs and requiring a kill switch. Nothing has been mandated yet, so the rating stays at 5 [6].

Sources

1. State of California, Executive Order N-9-26, September 18, 2026 (primary source). https://www.gov.ca.gov/wp-content/uploads/2026/09/FINAL-N-9-26-AI-EO-9.18.26-SIGNED.pdf
2. Office of the Governor of California, "Governor Newsom issues executive order to accelerate independent oversight and advance the creation of an AI kill switch," September 18, 2026 (primary source). https://www.gov.ca.gov/2026/09/18/governor-newsom-issues-executive-order-to-accelerate-independent-oversight-and-advance-the-creation-of-an-ai-kill-switch/
3. Office of the Governor of California, "Governor Newsom signs first-in-the-nation AI safeguards to protect Californians," September 9, 2026 (primary source). https://www.gov.ca.gov/2026/09/09/governor-newsom-signs-first-in-the-nation-ai-safeguards-to-protect-californians-calls-on-the-federal-government-to-do-its-part/
4. Recording Law, "Newsom's AI Order Seeks Kill-Switch Study, Not a Mandate," September 2026. https://www.recordinglaw.com/news/california-executive-order-n-9-26-ai-oversight/
5. The Verge, "Gavin Newsom is pushing for an AI kill switch," September 2026. https://www.theverge.com/policy/997516/california-governor-newsom-ai-kill-switch
6. Office of the Governor of California, "Governor Newsom announces world-leading experts to deliver on his AI executive order, including advancing creation of a 'kill switch'," September 23, 2026 (primary source). https://www.gov.ca.gov/2026/09/23/governor-newsom-announces-world-leading-experts-to-deliver-on-his-ai-executive-order-including-advancing-creation-of-a-kill-switch/