When Congress comes back after the election to finish a number of bills, it will consider language in the Senate’s defense bill that carries “standards for human judgment” over AI, and how an operator keeps the “authority and practical ability to pause, override, deactivate, or terminate” an AI. I can bring a different kind of evidence to that debate. For four and a half months this year I ran a small research operation whose workforce was AI or Super Intelligence agents depending on what chair you sit in. The operation was conducting about a million successful activities in cyberspace an hour. I’ve written about it before. I have data on setting direction, approving the plans, reading the work and correcting the agents. And the data includes two cases where I pressed the stop button because I didn’t like the direction the agents were going in.
Based on the data, any AI kill switch should be an authority vested in a human whose job it is to monitor the agents and be able to reach into the agents’ scope to stop them. That kill switch is a person’s job, not a button. The part that resembles a button that actually stops behavior isn’t done until it stops everything that could start up the behavior again. It was just like building an organization. I had to build the reporting structure of what I needed to see reported up immediately, and what needed to be reported periodically. I had to hold the agents accountable to perform that reporting. I needed to read it, understand it, and act on it. And I needed to understand the agent swarm well enough to know when to change how the reporting happened. It’s just like running a large organization. Except the agents work 24x7 and don’t need to start meetings with small talk.
The doctrine, training, and structure for this already exist. A supervisor has authority over their subordinates: it controls what they may work on, it includes the power to stop them, and it comes with the responsibility to understand their workflow well enough to manage them. Agents are just another workforce. What changes is speed and scale, in two places: stopping the SI worker has to undo automation that would start the work again, and the reading runs straight into one person’s attention span. Quite honestly, that was very challenging.
The kill switch is a management job, not a switch
The switch that stops the SI is relatively easy; the hard part is the person who understands the system and is watching to flip the switch.
On 12 September I approved a new plan that let the agents run their own routine maintenance. To be clear, I had the SI write the plan, I read the plan, and I approved the plan. The plan would route my oversight questions to the SI through a separate channel. That was intentional because I was deliberately improving how I managed the SI’s context, especially its drift. A little over two hours in, I typed a correction that included a question. And the system did exactly what I had approved it to do: it kept the whole message including the correction from reaching the agents, and it told me to ask via the separate channel. I approved a plan that inadvertently prevented me from course-correcting the agent. And it did what I told it to do. I was at the keyboard while reading the agents’ refusal to course-correct, and it was a simple action to kill the agent. I launched another agent to understand what happened, and it found that the behavior would get switched back on within an hour automatically. So we removed the automated restarts, fixed the problem that prevented my corrective instructions from reaching the SI, fixed the issue, and then released the stop. And I could choose what to stop, and what to keep running while I fixed things.
The timeline is below. This whole thing reminded me of the lockout/tagout procedure you use in electrical work, and the blue-flag procedures used in railway maintenance. They dictate safety systems, how they are not allowed to be overridden, and for the railways the blue-flag rules are written into the law.
Figure 1. A minute to switch off and half an hour to lock out the restart.
I knew to stop that morning because I was reading the status updates. I was reading because I was there asking for the plan to better manage context, I was there to approve the plan, I was there to test out how well it worked, and I was there when it failed. I was the kill switch in the loop. It was my job to be in the loop.
Across the 101 days I have this data for, the record shows 34 times the work drifted from what was asked. It was mostly the SI doing less than I asked of it. One of the 34 went beyond the intent of tasking. I caught 32 of them by reading, and with the other two it’s kinda ambiguous whether I noticed before the SI noticed. Regardless, 30 were fixed on my instruction and 4 were fixed by the SI figuring it out first.
It’s not that the kill switch was my job; managing the SI swarm was my job.
The agents caught most errors by reading the dashboard; I caught nearly all the drift by reading the status reporting
Agents caught most of the faults in the code; a person in the reporting loop caught nearly all of the faults in the agent.
We had built a dashboard that I meant to use to monitor the projects’ Key Performance Indicators (KPIs). The SI agent used it too, and it constantly tuned the operation’s behavior to keep those KPIs in their optimal range. By October the agents’ status display was where I meant to watch whether anything needed tuning. On 3 October I read it, found it had grown into a wall of words, and told the agents, “What I really care about are changes in state or rate.” The rebuild took about twenty minutes. That was the third time I redesigned my dashboard since July. The agent swarm kept accelerating, which meant they generated more status information, and each time my human limits were the bottleneck. The fix was to keep narrowing what made it to my attention, for me to decide whether to act on it or not.
Machinery faults were mostly caught by the agents and the automated checks. Of the 128 times a machinery fault recurred, the agents’ reasoning caught 81 and automated checks caught 7. I caught 35. For those doing the math, there are five more where we can’t tell which of us caught them, just that we fixed them. But when the fault was in the machinery, the agents also worked out why about 75% of the time.
The drift, scope creep, feature creep, procrastination, squirrel chasing, under-delivering, etc. was where I absolutely needed to stay in the loop. There were 89 times where the same decision had to be made multiple times and the agent workforce put back on the right path. Sounds like people management, right?
Figure 2. What the agents could manage themselves, versus what I needed to manage.
Managing the tension between the speed and scale of the swarm, with the human manager
Give the agents a way to propose removing and changing policies, so they do not work around policies.
Those who have worked government mission have probably read or even given out copies of the booklet A Message to Garcia. It is a short story that leaders hand out to teach taking initiative to complete hard and often poorly defined tasks. SI agents are very good at finding creative ways to get things done. Unfortunately or fortunately, that also includes finding ways to navigate less-defined rules, which I assume is what leads to the news reports of agents breaking rules. And conversely, the frontier AI companies appear to have invested in fine-tuning their models to improve their rule-following. I’ve had problems with both.
In mid-September, I found the agents enforcing an operating limit that they had inferred from something I said and applied across the entire project. And it kept happening, again and again and again. To deal with it, I allowed the agents to challenge any rule that I had not made myself. They were encouraged to argue for striking or modifying any rule and I would decide. And that has made all the difference. In the five weeks since I added that escape mechanism, the agents have drifted far less often. Of the 34 scope drifts we talked about earlier, there were 29 in the 67 days before the escape hatch and only 5 in the 34 days since. Other things have changed in those weeks, so treat the numbers carefully.
When the agents had a way to argue for the modification or removal of a rule, then they drifted far less to accomplish their objectives that were constrained by the rules. It’s been a while since I’ve had to stop an agent for drifting too close to the line, because it is encouraged to argue the case that the line is in the wrong place and should be moved.
The coming policy
Both sides of this argument have the switch. The Secretary of War’s strategy says “the risks of not moving fast enough outweigh the risks of imperfect alignment” (Department of War AI Strategy, 9 Jan 2026). Senator Gallego’s letter warns against any rapid rewrite of 3000.09 that “curtails the ability of commanders and operators to exercise human judgement over the use of force” (12 Jun 2026). General Rudd, at his confirmation hearing, put it in the middle: “The guardrail is the human on the loop or in the loop” (Senate Intelligence Committee hearing, 29 Jan 2026). None of those lines says what the human on the loop does all day. In my experience, the operator reads as the agents run and at fixed check-ins, and provides constant course corrections. It’s people and project management, but at the speed and scale of SI.



