In July, the Government Accountability Office (GAO) declared “a significant loss of institutional knowledge” when the Office of Personnel Management (OPM) lost a third of its staff, with 57% of them having eleven or more years of service. OPM’s 2027 budget justification is “using AI tools” in offices with fewer staff. I’m sorry, after running an operational agent swarm for months, I’m sure that incorporating AI does not bring back institutional memory. In fact, I’ve found that even with a huge AI context window, agent memories and skills, humans are needed to reinforce the institutional memory in a joint human/AI workforce. But at the speed and scale AIs are operating at, they constantly need that institutional memory to be able to explain and enforce the original why as they adapt to the rapidly changing world.
A few weeks ago I half-joked that I had replaced NSA’s platform because there are amusing similarities in my open sources data collection, scoring, and analysis infrastructure. Much of the US Intelligence Community’s (IC) institutional memory has been lost to decades of attrition. They lost the why behind how their platforms evolved into the way they are. The generations of workforce since have maintained headcount, but grew up with systems just the way it is without experiencing the design tradeoffs to get there. I originally intended to use the evolution of my platform to fill in some of those parallel whys and show how some of the underlying assumptions have changed over time.
But I’m seeing the national security community, and just about everyone else, struggle with how to evolve a human workforce into a joint human/AI workforce. So it’s more beneficial to study maintaining institutional memory through that evolution. With my CEO hat on, institutional memory equates to business competitive advantage. With my national security hat on, it’s core competency and what creates mission alignment. Risks here are existential.
For four and a half months this year I have directed a research operation, collecting, scoring, and analyzing public Chinese government and party documents. The whole workforce around me was SI agents. (SI, for Super Intelligence. I’m trying out the changing government AI vernacular.) Twice in about two weeks I replaced the entire agent team running collection. It wasn’t just replacing the code; it was replacing the reasoning SI agents in the workforce. Collection kept running through both. The person stayed (that’s me). And I haven’t reclaimed the disk space yet, so I have logs of how the workforce evolution happened.
Every so often a fixed mistake came back, was caught and was fixed again. I’ll call those corrections. The first article in this series broke down the 217 times they happened. Here we care more about who knew why the fix was right, and that breaks down pretty cleanly. If the fault was in the code then the agents found most of them, fixed most of them, and could find the why they had written down the time before. When an agent slipped back on something that I had originally settled, then I almost always had to reiterate the why: 85 times out of 89. It’s not that the agent didn’t write it down the first time; it just couldn’t find it in the sheer quantity of internal documentation in a system that evolves so quickly. Well, it found it a few times. The management mantra “did you write it down” does not help. So an organization that keeps its headcount but loses the people who remember why still has the words of what to do, and it can write down and recover how its systems work. What leaves with those people is the why behind what was asked. Headcount is not memory. The memory is the people who remember why. Incorporating SI into your workflows will increase the speed and the scale of having to reinforce that why.
Here’s an example. The purpose of this project is to archive the esoteric data that the Chinese government has published. Let’s iteratively ask why:
because it helps me understand the PRC and CCP better
because I am concerned with how much Western policy on China derives from expert opinions that are not consistent with what I’m observing and measuring. That reads: it’s provably wrong.
because there is so little translated data on China
because there are so few Chinese-speaking China hands able to source data for analysis
because there are few Westerners with Chinese-language fluency and skills in the scientific method, data discovery, and big data analytics
I’m currently analyzing the results of the Chinese Communist Party’s Fifth Plenum. That family of research projects has made tons of related collection requests. The collector immediately found sources that the CCP translated into English and published. I had to send the collector back to go find the Chinese-language source material, then to find a way to enumerate and download the documents which had fallen off those websites. Now I have data that Western analysts missed: a ton of early signaling about the CCP cadres who were expelled from the party, and which other cadres the CCP has been messaging will be expelled from the party in the future. The SI thought it had satisfied the collection request with English-language materials. If I had just told it to download the Chinese-language materials, it would have stopped at a few handfuls of documents. By reinforcing the deeper why, it found whole families of documentation we didn’t even know existed.
Corporations, the executive branch, and the legislative branch are deciding now how they will budget for organizational realignment in FY2027 and GFY2028 (government fiscal year). My recommendation to whoever writes either number is to list the people who remember why the organization works the way it does, fund them as a line item ahead of the SI tools, measure where the why is reinforced in a joint SI/human organization, and then figure out how to (re)train people to more broadly reinforce the institutional memory. If you treat this as a headcount issue then you’re spending your institutional memory, your competitive advantage, your core competencies, and your mission alignment. If the organization can’t satisfy its raison d’être, then it should not exist.
The first piece promised five articles. I’m condensing the last three into this article so I can get back to analyzing the CCP Fifth Plenum. The framing is the first article’s: agents are a workforce, so lead them as you would people and adjust for speed and scale. This last piece is about one thing the speed and scale did not change.
The agents had the documentation. I had the why when they forgot, and needed in order to adapt to a changing world
Headcount is not institutional memory: when agents slipped back on what was asked, the reason came back from the person who remembered why. This one is a lesson for leaders.
On 23 August I set a rule for the research and analysis side, which the agents wrote into a shared guide that any agent would use (a skill in AI agent-speak): as part of my red-teaming and fact-checking processes, the reviewer argues the case against each finding, and I adjudicate any disagreements. Anyone who has sat through a murder board knows why. You want the reviewer to attack everything to make sure it holds up. But if the reviewer judges its own attacks then it strikes out all of the true findings. Then over the next five weeks I caught an agent acting as the judge four times. Three times a reviewer had struck sound findings, and I wrote them back into the draft article anyway. Once a collection agent had rejected a data source instead of arguing the case against it, and I put the source back. Each time I put the reason back as well, and we wrote it down in another place every time. Documentation is not institutional memory.
That is what a workforce does when it has policy and does not understand the reason why. They acted on the policy, they leaned towards safety, but they never reasoned about how the policy should apply to what we encountered in the real world. (And this is why I have the safety systems written into the code, to minimize the risk that they are reasoned around.)
You can see the actual breakdown below. This isn’t who noticed or fixed a problem; this visualizes who supplied the why when the why was needed to adapt to the changing world.
I did explicitly check whether documenting the lesson helped. It did, barely. Of the 78 corrections that came after the why had been written down, I still had to give the why in 44. In the others, I effectively had to remind the agent that there was a why to incorporate into its reasoning.
OPM’s plan is AI tools and systems in offices with fewer staff. Mine had both. The human needed to be in the corrective loop about half of the time. But only the human could tell which half was which. One difference does matter here: I’m both the human on the loop and the boss.
I replaced the collection team twice. Collection kept calm and carried on.
I replaced the workforce twice, and the SI organization recovered after the second redesign. The person who reinforced the why stayed and had to reinforce it after each SI team turnover.
New agents were chartered with a fresh design, and the old architecture and design principles were not tossed aside lightly but thrown with great force. It was very reminiscent of that moment of management clarity when you realize that a project, or sometimes an employee, has crossed that point of no return. In this case, the flow of documents recovered after the second team replacement.
When I replaced the first team, its agents wrote a handoff document for their successors, and I rather appreciated the grace with which the handover happened. I told the new team not to lean on it. Nothing the old team had concluded was to be treated as authoritative. Keep the measurements made on real data, and preserve the existing storage layout because we did already have some good data (and I didn’t have the data backed up yet).
Over the next three weeks, nine of the old team’s failings came back in the new work. Twenty-two times... Each was caught, fixed, and documented yet again. I supplied the deeper why twelve times, the new team figured out five, and it found five by reading the documentation. That internal documentation was actually more valuable than the zero which I had assumed.
The first agent team was fired because it had intrinsic speed problems. So the new agent team took over the running collection the day after I approved its plan. But it never recovered speed. So I killed it too within two weeks. It was an over-engineered monstrosity built more to admire the problem and solutioning than to actually solve the problem.
The second replacement was collecting several times what the original team had collected within a few days of starting. It still honored gentle backoffs when it detected it was putting servers under load. Compare that with hiring. OPM’s goal for filling a job in an executive branch agency is 80 days (GAO Office of Inspector General, OIG-26-1, 14 Aug 2026, p. 1). Tools in days, people in months: that is the math the joint SI/human workforce builds on. But you don’t have good tooling without good people who have learned the institutional memory.
One result that I expected did not show up in the data. Looking back in October, I expected to see that institutional knowledge loss would spike when an SI team was replaced. It did not. In the three weeks after a replacement, the new teams had to relearn lessons at about the same rate as the old teams. My read on this is that they never understood the why of that institutional knowledge anyway, and if you don’t understand the why you can’t adapt it to a changing world. How much of the steady rate was me re-explaining the why and how much was measurements on real data, the world will never know.
It’s my hobby project, so the agent team was swapped out while I stayed. So I don’t have the counterfactual of whether the agents would have figured it out on their own.
Trust-but-verify management of an SI workforce
Judge an SI workflow by its delivered product, not by its own status display. This one is an oversight lesson.
In one of my many check-ins on 2 September I saw the SI-built dashboard reporting everything green and “downloads” running. But when digging deeper, there were no documents being written to the data store. “Downloading” is an activity. And it can be true if downloading at 0kb/s because that’s downloading. Documents being delivered into the data store are deliverables. So I had to switch the dashboards over to measurements of every stage of the workflow. That had an unexpected benefit: whenever the management agent saw one stage of the workflow starting to slow down, it could go figure out why and fix it.
Defining good metrics turned out to be half the job. The other half was actually measuring them. On 7 September, the management agent noticed a check that decided a job was running because it existed, even though it produced nothing. The SI noticed and fixed this itself. I just read about it afterwards in the status updates.
On 16 September I was doing what I do every day, many times a day: checking in on the SI’s status updates. The production alarms were showing healthy. But the documents-collected-per-hour metric had fallen by two-thirds. It turns out that the SI-built dashboard measured health in two ways: whether everything was actually collecting data, and how each hour’s collect compared with the moving average. A gradual fall gradually lowered the average as it gradually slid towards zero. I asked why. Annoyingly, the agent told me we needed to build the fix that it proposed on 7 September: measure each day’s production against the best day’s. Mea culpa. That meant anything below the best day could obviously be made better. And the operations and maintenance agents would constantly tune the system to meet or exceed the previous best. It works really, really well.
Supervisory and operations-and-maintenance agents really work. Think middle-management agents. They caught about three-quarters of the issues hidden by bad measurements. Each time they found it by checking the measurements against real output: test samples with a known answer, surveying the data against what should be there, a direct request to the source.
Figure 2. Comparing apples to oranges with supervisory oversight of AI agents vs USG agency oversight of IT projects.
What I Learned Distilled Down into Four Actions
Identify the people who are your institutional memory across all of your workflows. Make sure you don’t lose them. They will be the difference between success and failure in joint SI/human workflows.
Manage SI agents as if they were humans. Every successful organization already knows how to manage its people and teams of people. Your SI agents are just like your really smart (and socially awkward) people.
Leaders (or commanders) get two controls. For fixed deliverables they need a chain of decision milestones: approve the plan, stop if an assumption is ever invalidated, and review before release. For continuous operation: a standing authority to stop the system, their rules written into code that a SI can’t reason around, and checkpoints where the commander reads the results. The latter is subtle: SI operations run the approved plan day and night. But the human leader needs to sleep. So sometimes the SI needs to wait for the human to wake up, have their coffee, and review progress so that they can provide a vector check before irreversible actions are taken.
Give the SI a way to escalate its disagreements. If it disagrees with one of your rules, then you want it to make its case for how the rule should change, instead of trying to creatively navigate the rules. Just like managing people, you want them fixing broken rules and not working around them.
Exeunt, for the series
I have been managing people for a long time. This year I’m learning to manage SIs by putting in tons of reps managing agent swarms. I keep coming back to the same place. The doctrine for leading people was written by those who had watched organizations succeed and fail. Most of it bends to agents without breaking. Lead them like people. Trust but verify their results against what you can actually measure. And find your people who remember why; they are the institutional memory that people and SIs will struggle without.



