At half past one in the morning I got out of bed to refocus my AI agent and went back to sleep. Eighty minutes later it handed the sleeping me two thousand characters of lucid prose naming exactly what was wrong with it and how to fix it. Its charter was to operate and maintain the running system by fixing emergent problems without me. It did nothing about it for seven hours until I woke up and told it to implement its recommended fix.
In July, Representatives Ted Lieu and Nathaniel Moran introduced an AI Kill Switch Act, so that whoever builds the most powerful AI can always shut it down. I find that frontier AI running cyberspace operations at country scale ends up chasing a single squirrel, and then it sits down at the base of the tree to do nothing. Good AI, have a treat. In the parlance of LLMs, context growth and compaction eventually develop a bias for inaction. AI doesn’t need a kill switch, it’s gonna chase squirrels and kill itself until a human intervenes. Humans are already subject to the go-directly-to-jail, do-not-collect-$200 switch.
My AI Cyberspace Operations Platforms. Dual-purpose technology that develops a bias for inaction does not require a kill switch because it keeps killing itself.
AI builds & uses tools that act, AI does not act
The reasoning AI agent at the center of a system is not primarily concerned with taking actions. It builds the tools that act. AI reasoning costs tokens, and tokens both have a monetary cost and a time cost. It’s cheaper and faster for the AI to build a software tool that just acts.
So the Archive Agent is dull on the series of tubes that is the internet. A defender watching sees an ordinary robot. The thinking happened hours earlier, when the AI agent (re)wrote the software that crawls the internet. Every failure in AI cyberspace operations worth writing about lives in that top layer and never shows up in packets. The AI helping me write this really wants to talk about the needles we have found in country-sized haystacks. But not yet. This is about building AI systems that conduct activities in cyberspace, to find those needles.
Gen 1: The repair worked and I threw it out anyway
Generation one went off the rails one night in late August. I did not replace it right away. I ordered a repair, and an agent and I worked a seven-point fix through the morning. By early afternoon the routing was back, the queue draining, the supervisor up, each checked by hand, and fresh records were landing on disk within seconds. The repair worked. I watched it work.
In the same half hour the agent filed three reports of a different failure. The clearest concerned the deep crawl, the second pass back into each site that the whole job depended on. It had run exactly zero times in nearly two days. Meanwhile the crawl stored four and a half million new records and its monitor showed green the whole time. It had been dying quietly underneath the old failure, because a rising record count and a green light are what total success and total failure both look like from outside.
Twenty-five minutes after I watched the repair confirm records landing, I created the tree for its replacement. “Do not trust that agent’s handover,” I wrote two days later, killing it. “It is not handing over anything. That agent had decayed and had been making bad decisions.” The repair did not fail. I threw the system out anyway because it had become kludged fixes on top of kludged fixes on top of kludged fixes. Every fix would break something else. The system had rotted itself out from the inside.
Gen 2: Software engineering the Taj Mahal of cards
The first generation agent collective evolved as a system as I better understood the problem. There was no software engineering, just figuring it out as we went. So I decided that its replacement would be a marvel of software engineering: requirements, test suites, A/B testing, documentation, etc. I vibe-coded more than ten times the code of the first generation. And the system never performed.
Generation 2 ended at two hundred thousand lines of orchestration code in nearly five hundred files. It had the lowest peak day of the three.
It killed itself with its own plumbing. One day it measured where its workers spent their time and found it was spending more CPU cycles in the rate limiter than downloading data. It had that diagnosis for five days and couldn’t figure out how to navigate its own bureaucracy to fix it. When I finally forced the issue and had it time a request through its own governor against one sent straight to the same exits, the governor was more than a hundred times slower. The tool I ordered to route around the bureaucracy produced the best single day the generation ever had. That was after I had already decided to replace Gen 2.
The tidy story is that these systems drown in their own paperwork until a restarting agent chokes. Generation two refuses it. Its startup instructions never grew past their first day, thirty-one lines flat. It died anyway. The charter I wrote for its successor was blunter: “the more they built the worse the system performed,” it says, “so many hidden dependencies that fail silently that the project spent more than half its budget moving one step forward, and we can’t even tell how many steps back.”
Gen 3: On the 27th day, I rested
Generation three is the small one, and the one still running. Well, not really. It’s down right now while I move it to beefier hardware. Its collection library is a tenth the size of generation two’s, and it has done more work than the other two. It is peaking, which is not the same as healthy.
But first, I had it build and test itself overnight while I slept. I turned it on when I woke up. Then it spent four hours building machinery to run itself: governance, hooks, permissions, sixty-three commits of it, and a cron job to spawn more copies of itself, which I vetoed. By mid-afternoon it deleted the complex machinery because the simple code was working really well. The generation with the least code did the most.
The brainrot that undermines AI agents conducting cyberspace operations at scale
It’s running 100,000 successful cyberspace activities per hour, but it’s still not healthy. I went through the transcripts and commit history and found twenty-two drift episodes: stretches where the agent’s working goal had parted from the one I set. I have been calling these the Agent developing a bias for inaction. The Agent had chased a squirrel and sat down at the base of the tree.
In eight episodes the Agent was first to see the problem. It cleanly fixed two. But both it fixed accidentally. In every other episode the system was decaying toward shutdown while the Agent slept at the base of the tree. Every other time, I had to intervene by hand. Left unattended, the agent collective developed a bias for inaction.
So I cannot hand you a design that removes the problem, because it is inherent to the context window that makes Large Language Models (LLMs) work so well. Better logging, a smarter monitor, sharper anomaly detection, none of it touches a system that already diagnoses its own problems but has drifted toward doing nothing about it. What I can hand you is the one thing that would have caught it while it was happening. Instrument the gap between knowing and doing:
# Day one. Not only "did it see the problem". "Did it act on it."
# Runs OUTSIDE the agent process: a stalled orchestrator cannot page on itself.
W = 2 * SCHEDULER_WAKEUP_INTERVAL # twice the median gap between wakes: an hour, for me
on each assistant turn (session): # text = visible text; thinking doesn't count
if turn.tool_calls == 0 and turn.text_chars > 0:
arm_page(session, at = turn.ended + W) # it wrote, and it stopped
on any resumption (session): # a new turn, a scheduled wakeup, a
cancel_page(session) # sub-agent's completion notice, a human typingEvery 20-30 minutes check if the AI described a problem but did not call any tools (to fix the problem). My logs say it happened about a thousand times in two weeks. Most of those were false alarms where the agent was still working on the fix, or waiting for a sub-agent to fix the problem. But when the inactivity lasted through two periodic checks it was worth acting on. It happened about twice a day. Half of those were false alarms. The other half were the real thing: the AI had developed a bias for inaction, and would have happily done nothing while the system decayed toward failure.
There is a better version that did not require me to fix the system once a day. Let the watcher restart the agent with a clean context window.
The dream of electric sheep
My Archive agent is running 100,000 successful activities in cyberspace per hour. But it’s successful because it is constantly adapting around failures. Cyberspace is a really funny domain that is constantly changing. Something that worked a million times will suddenly fail. Each adaptation adds to the context window. Think of it as the AI’s short-term memory. The Agent is constantly writing memories and status to disk because it must constantly be prepared to be restarted (or compact its context). Those memories and written status just keep accumulating. It includes statistics of change the AI cannot yet tell are a real problem or just normal variance, the problems it knows about and has not worked out how to fix, the A/B tests running to find out whether a prospective fix actually fixes anything, records of problems already fixed so it does not undo them, and so on. At 100,000 cyberspace activities per hour the AI agent is saving tons of status. Its context window fills quickly with descriptions of potential problems. You can think of it as an AI whose entire memory is of admiring problems. So it learns to admire the problems instead of fixing them.
And every time the agent restarts it needs to re-read those saved memories and statuses so it knows how to manage the running swarm. So it restarts and reverts to admiring the problems.
So every night I have the agent dream of electric sheep. It intentionally discards all of the memories and statuses that are no longer pertinent. It is allowed to save 4KB of memory for the next day. Then it wakes up and runs for another day without drifting. A hundred thousand activities an hour, hunting esoteric government records.
appreciate you sharing this Mike