Who Would Hold California’s AI Off Switch?
Under N-9-26, signed on 18 September 2026, Government Operations Agency must recommend potential safety-law amendments by 16 November of that year, consulting Cal OES and specialists across the country. The proposals include frontier-model shutdown controls; they are not an immediate developer obligation.
The order starts a proposal process
Officials must assess feasibility, efficacy and ongoing testing by independent organisations. The order sets no national rule, final engineering standard or liability settlement.
A shutdown control is a system boundary
Turning off one public endpoint may leave copied model weights, private instances, cloud jobs or connected agents running elsewhere. A useful control therefore needs a declared scope: which service, model, credentials, tools, data pipelines and compute paths it can stop.
Assurance needs more than the operator’s promise. A test should show that the control works, that evidence survives the stop and that recovery follows an authorised process. The control itself must resist misuse, political abuse, accidental activation and denial-of-service attempts.
Questions the design must answer
- What triggers action? The threshold should use observable evidence and a defined severity test.
- Who authorises it? Emergency speed needs to sit beside due process and an audit trail.
- What stops? APIs, model serving, tools, credentials and downstream agents may need different controls.
- How is it tested? Independent exercises must demonstrate a working stop without opening another attack path.
- What survives? Logs, forensic evidence and safe recovery functions need protection during a shutdown.
- How does service return? Restart authority and validation are part of the safety design.
For organisations planning compliance, document containment boundaries and testable recovery now, while keeping the legal distinction clear. A defensible stop must not become an easy sabotage route.
