The Secret the Government Keeps From Itself

An intelligence assessment of the US classification system, by the rules the intelligence community itself uses

I had a NSA security indoc years ago where the security officers distilled the rules down quite plainly: if you get it wrong then you go to jail; if you don’t report it then your coworkers will report you for it and you go to jail; if you don’t narc on your coworkers then someone else will, you lose your job, and probably go to jail. But no one ever describes what happens if you overclassify something because that is the behavior being incentivized. The 9/11 report was pretty damning about it. I took what I expected to be a three hour cruise through the data to see how bad the over classification problem is.

A scope note, because this is not a normal policy piece

I tried to audit American over-classification the way you would audit any system you expect to have been gamed: Inspector General reports, classification-cost ledgers, appeals-board dockets, clearance-revocation decisions, federal award archives, the patent office’s secrecy-order lists, a benchmark of AI models against signed classification rules. Not opinions about secrecy. Paperwork. Evaluate the n-th order effects because it takes quite the mastermind to game the system and cook all of the side effects out of others’ books.

Halfway through I had to change my method. The security apparatus controls the records that would prove or acquit it. It does not publish the audit. It stopped publishing its own cost. Its rulebooks (security classification guides) are themselves classified even where the source documents have been declassified. That is not a research inconvenience. That is the structure of a denied area, and the right way to write about a denied area is the way the intelligence community writes about one: as an assessment, with confidence levels attached. So I have two intelligence assessment research projects in flight right now: one on finding and evaluating embarrassing data that China hides, and one on finding and evaluating data the US national security community hides. Is it embarassing to them?

So that is what this is. When I say high confidence, multiple independent lines of evidence converge and I would be surprised to be wrong. Moderate confidence means the evidence is credible but single-method or carries a known gap. Low confidence means the judgment is plausible and internally consistent but rests on something unmeasured. And here is the part policy readers should not skip: a low-confidence judgment is not a shrug. In the intelligence world it is a collection tasking order. It tells you exactly what to go collect. The end of this piece is a government collection plan to provide civilian oversight of the national security establishment.

The research data was current as of July 2026. Dollar figures are US federal budget dollars unless noted.

Bottom Line Up Front

How to read it: fourteen judgments, color-coded by confidence, with the mix of evidence behind each. What to see: the mechanism and the unauditability carry high confidence; the exact rate and cost are the part the government would have to measure itself.

Here is what a decision-maker should now believe.

The incentive facing every American classifier almost certainly punishes under-protection and never punishes over-protection. That is no longer testimony. I searched eight public discipline channels spanning roughly forty-seven years, including all 32,213 published industrial clearance decisions, and found zero sanctions for classifying too high, against 163 career-ending sanctions for protecting too little in the same corpus. High confidence.

When an independent, fully cleared referee re-examines a contested over-classification, the classifier loses three times out of four. The appeals panel the system built for exactly this purpose overturned 75.2 percent of the contested decisions it reviewed across twenty-two years. Contested cases are a selected sample, and I will say so again below. But the direction is not subtle. High confidence in the record; the population rate remains unmeasured.

How much of the classified pile never needed protecting? The honest answer is a range that the government could collapse any time it chooses. The provable floor is 3 to 8 percent, and that floor rests on auditors who could strictly adjudicate only one-seventh of what they reviewed. Every official and insider who has ever put a number on it, from 1956 to the ISOO (Information Security Oversight Office) director himself, said half or more. I assess with moderate confidence that the true rate sits well above the floor, and I put the documented range of expert belief, not my model, on the chart.

The cost is a slope, roughly 12 billion dollars of budget per unit of excess-classification share, and where you sit on it is a belief about the one number nobody measures. The innovation damage is not a belief: prolonged compulsory secrecy destroys 20 to 36 percent of follow-on value, rising past half for the longest orders, and that result now survives every matching design I could throw at it. High confidence.

And there is a reform whose core does not depend on measuring any of this, because its job is to produce the measurement. Most of it can be done by Executive Order and does not require legislation. I know, still a high bar.

1. The bet is lopsided, and now it is counted

The classification system asks millions of cleared people, tens of millions of times a year, one question with wildly asymmetric stakes. Leave something open that turns out to matter and it can end your career. A revoked clearance is not a bad review. It is expulsion from the field that pays you tens of thousands of dollars a year more than comparable uncleared work.

Protect something that needed no protection, and nothing happens. Not rarely. Never. The census: Merit Systems Protection Board decisions, Office of Special Counsel reports, the Inspector General corpus, Defense Department and Department of War administrative releases, 32,213 industrial clearance decisions across thirty years, Government Accountability Office records, and the sanction-reporting sections of thirty annual oversight reports. Zero recorded sanctions for over-classification, in any public channel, ever. The two formal complaints in the record were both dismissed. Meanwhile the same clearance corpus contains 163 sustained mishandling sanctions, and where the mishandling allegation was proven, 110 of 111 people lost.

How to read it: each channel searched, with sanctions found for under-protection versus over-protection. What to see: the zero is not an absence of data. It is the finding.

I began this project believing the driver was money: security officers protect the salary by over-protecting the documents. The model says that is not it. If you sanctioned over-classification with the same career stake, the stake cancels. What is left is the difference in the odds of getting caught, and one side of that ratio is now a measured zero in every public channel. The formal rules authorize sanctions in both directions. The realized practice runs one way.

The formal overclassification correction channel confirms the flow. The rules invite any authorized holder to challenge a classification they believe improper, with formal protection from retribution. Relative to the tens of millions of classification actions a year, formal challenges are almost certainly vanishingly rare. Why is a question the public record cannot answer. I assess with low confidence that anticipated career consequences do the suppressing, the rates are unmeasured, and the protected practitioner survey that would settle it sits in the collection plan below.

2. When a referee finally looks, the classifier loses three times out of four

The system has an independent referee. The Interagency Security Classification Appeals Panel (ISCAP) is cleared, sees the underlying Security Classification Guides (SCGs), and its decisions cannot be reversed by the agency it overrules. It is the closest thing that exists to an incentive-neutral second opinion.

From 1996 through 2017 it re-examined 2,935 contested documents and sided against the classifying agency on 75.2 percent of them, releasing 31.4 percent entirely unclassified. The rate held in every era and rose in the most recent one. Even the agencies’ own internal declassification reviews, judging themselves, release about nine requests in ten at least in part.

How to read it: the share of contested classification decisions overturned in full or part by the cleared, source-access, non-reversible panel. What to see: when the incentive is removed, the judgment reverses three times out of four.

The caveat, stated plainly: these are contested and aged documents, the cases somebody cared enough to appeal more than they cared about retribution from security staff. They do not measure the whole pile. What they measure is what happens to a classification decision when the career stake is removed and someone qualified actually checks. That is precisely the experiment the incentive story predicts, run 2,935 times.

3. Everyone who ever put a number on it said at least half

You have probably seen the claim that the government over-classifies about 65 percent of what it protects. In my own earlier work that number was an assumption. I have now done what should have been done first: collected every documented numeric estimate of the genuinely-sensitive share ever made by an official or practitioner in a position to know, twenty-seven sources, fourteen of them numeric, from a 1956 Defense Department review to the 9/11 Commission’s chairman to the man who ran the government’s own classification-oversight office, who told a Senate hearing the over-classification rate was, in his words, beyond 50/50.

Every at-creation estimate since 1956 puts the excess at half or more. The distribution is honestly bimodal: senior officials cluster near 50 to 55 percent, career insiders and document-level reviews cluster near 90. The weighted median of documented belief is 83 percent, with a 90 percent band running from 38 to 99, which is the honest width of a question nobody has measured. My old 65% sits in the saddle between the camps, and a deliberately skeptical weighting reproduces it almost exactly.

How to read it: each point is a named official or study, with the basis of the estimate; the line marks the median of documented belief. What to see: not one informed person who committed to a number ever said less than half.

This is an elicitation, not a measurement, and I hold it at moderate confidence for exactly that reason. But the burden has moved. The claim that over-classification is rare is now the claim that every official who ever went on the record about it was wrong in the same direction.

4. Nobody can measure it, and the government has stopped trying

I hit the measurement wall four independent ways, and the wall is the finding.

When Congress ordered every classifying agency’s Inspector General to measure over-classification directly, the resulting audits could strictly adjudicate 523 documents out of the 3,797 they reviewed, one-seventh of their own sample. Their provable floor, 7.8 percent, describes that sliver; allow the unadjudicated mass to err at anywhere from zero to twice the proven rate and the floor itself swims between 1 and 15 percent. Not one of the thirty-three reports produced a representative, independently adjudicated rate. The Intelligence Community published no rate at all.

I tried to audit declassified documents against the actual signed classification guides they cite. Of 145 public documents naming their guide, the number whose guide was also public was zero. The rulebook itself is classified, so the rule can never be checked from outside.

The oversight office that once published what the classification enterprise costs stopped in 2017, after telling the Senate it was not confident in its own last number. My reconstruction puts the enterprise very likely at 23 to 29 billion dollars in 2025, a judgment I hold at moderate confidence because it is anchored to the last official figure and bridged forward, and about 92 percent of it no longer corresponds to any public budget line.

How to read it: four different measurements of four different things, deliberately kept apart, from the provable floor to documented belief. What to see: the famous 70 percent marking-error statistic is a formatting construct and belongs to none of them; anyone quoting it as an over-classification rate is pooling things that must not be pooled.

A system built to resolve every doubt toward secrecy is, by construction, a system that cannot audit itself. I assess with high confidence that the true rate cannot be measured from outside, and with moderate confidence that no agency currently measures it inside either. If a favorable representative audit existed, publishing its aggregate would cost nothing, would answer people like me, and it would reign in the security apparatus that has confused security for National Security. The audit has never appeared.

5. The bill is a slope, and here is how to read it honestly

Every cost channel scales with the same unmeasured quantity, the excess share, so the honest cost object is a slope: roughly 12 billion dollars of budget per unit of excess-classification share. The base it multiplies is no longer an assumption. The Pentagon’s own budget exhibits carry explicit classified-programs lines totaling 44 billion dollars in 2019 rising to 62 billion in 2025, a steady fifth of the Pentagon’s acquisition spending.

Read the slope at the audited floor and the excess costs about 0.9 billion a year. Read it at the modeled center and it is about 4 to 5 billion. Weight it by the documented beliefs in the chart above and the median is 9.1 billion, with an honest band of 4 to 14. Only about 60 cents of each dollar is real resource loss; the rest is transfer. A budget hawk and an economist should read different numbers off this figure, and both should say which one they are reading. To be explicit about my own confidence: the slope I hold at moderate confidence, the level at low, because the one number that picks your point on the line is the number nobody measures. Low confidence on the level is the data collection tasking.

How to read it: cost rises with the excess share; the markers show the audited floor, the model’s center, and the median of documented belief. What to see: there is no single cost. Your belief about one unmeasured number picks your point on the line.

I will also say what did not survive my own checks. Clearance-required Requests for Proposals (RFPs) solicitations do not draw measurably fewer bidders at advertised competitions. Classified-intensive weapons programs show no average performance penalty across 651 program-years, though that sample by construction excludes the most classified programs. What survives is narrower: work that migrates into closed task-order channels carries a 6 to 9 percent price premium, a premium that could partly reflect vendor specialization rather than classification since that separating test remains unrunable, and the failure tail is ugly in a way averages hide.

6. Long secrecy destroys most of what it delays

One channel I can defend with real identification. Under the Invention Secrecy Act, patent applications judged sensitive are barred from disclosure. I matched 3,141 patents whose secrecy orders were later rescinded against roughly 49,700 controls granted the same year in the same technology class, and compared forward citations from the moment both were public.

Secrecy orders shorter than three years are a built-in natural experiments, since ordinary patent pendency ran two years and a short order delayed nothing. The natural experiment shows zero deficit, in every design. Then the response climbs: roughly 25 percent of follow-on value gone at three to five years under order, 40 percent at six to ten, 60 percent beyond ten. Pooled, the deficit runs 20 to 36 percent depending on how strictly you match, and matching patents by the actual text of their abstracts lands at 33 percent with the treated patents proving topically ordinary. Knowledge released late is not delayed value. Most of it is destroyed.

How to read it: forward-citation shortfall by order duration; the shaded band spans all matching designs; the short-order placebo sits at zero. What to see: the placebo passes and the damage grows with the dose.

I hold this at high confidence as a well-identified association. And the institutional trend runs the wrong way: the stock of secrecy orders is at an all-time high, while rescissions have collapsed from hundreds a year in the early 1990s to a few dozen.

How to read it: the top lane is the number of secrecy orders in effect each fiscal year; the bottom lane is how many were rescinded that year. What to see: the release valve rusted shut while the pile grew to a record.

7. The security pocket veto is real; its frequency is the intelligence gap

What security never says “yes” it becomes a “no” whether that is a deliberate pocket veto or just an overworked security team is something we can look at. There are six documented episodes that pass a strict four-part test: an upward security action, an incompatible channel, a linked operational effect, and a later competent review that judged the block wrong. The Army cleared a book called Operation Dark Heart; a late intelligence-agency override led the Pentagon to buy and destroy 9,500 printed copies; the Pentagon’s own re-review then lifted 198 of the 433 disputed redactions. Another manuscript sat frozen until two weeks after the author sued, at which point 85 percent of it was released. The pattern repeats back to the Pearl Harbor intercepts.

The silence channel is now quantified too. In 1982 the CIA answered manuscript reviews in 13 days on average under a 30-day clock. By 2016 its own inspector general projected book-length reviews at more than a year. Submissions grew from 43 a year to more than 8,400. The clocks remain legally non-binding, so silence still functions as a denial that no official ever has to own.

How to read it: each case, the blocking action, and the outcome once a competent authority looked. What to see: in every documented case, the eventual review sided against the block.

Existence: documented, moderate confidence. Prevalence: unmeasured, low confidence, and that is a tasking, not a concession. The registry that would count these events is specified below.

8. What AI failed at, but it failed forward in a way we can use

I benchmarked four AI models as derivative classifiers against 879 signed classification-guide rules. All fail the accuracy bar the reform’s own math demands, the best at 94.4% against a required 98.6%. All are anti-calibrated, meaning they’re about as confident wrong as right which kills model confidence as an “escalate to a person” trigger.

Two deeper results matter more than the failure. Judging blind, the models classified 42 to 64 percent of a genuinely all-unclassified guide’s topics. Handed the correct security classification guide, that reflex collapsed to essentially zero, in every model. It could decide unclassified. An uncertain adjudicator hedges toward secrecy exactly like the human the incentive model describes, and the right classification guide removes the hedge.

But hand it the wrong classification guide and the reflex averages 73 percent and never falls below about 53, and grounding in an all-classified guide is worse than no guide at all. The AI models anchor on the label distribution they are shown. So the honest engineering claim narrowed: the tool disciplines the reflex only when retrieval hands it the correct SCG, which means guide retrieval itself must be part of any accreditation.

The critical insight is that the models’ disagreement is informative even though their confidence is not. If you make the AI models think about it and estimate a probability that they’re right, you the average their own probabilities, and if there isn’t enough consensus in confidence you elevate the decision to people. The system auto-decides 86 percent of the workload while wrongly answering 2.16 percent of what it decides, escalating the hard 13.78 percent to people. Truth in statistics: that 2.16 carries a confidence interval of 0 to 4.93, so a guarantee that holds at the interval’s edge costs roughly 60 percent escalation on my benchmark, and the whole result assumes the tool was handed the correct guide. I built the proof of concept in two hours during the USA-Belgium World Cup match so some dedicated engineering with real test data will likely perform far better.

How to read it: the rate at which models wrongly classify genuinely unclassified topics, by what rulebook they were shown. What to see: the correct guide erases the secrecy reflex; the wrong one makes it worse than guessing.

How to read it: the auto-decide miss rate as more cases are routed to a human, with the 95% confidence band shaded. What to see: the committed point is 2.16% missed at 13.78% escalated, but the interval reaches 4.93%, and holding the miss at 2% across that interval would cost about 60% escalation.

The reform: relocate the no, and make it produce receipts

If the disease is a rational officer resolving every doubt toward secrecy for their self-protection, the cure is to reprice the doubt. Today, clearing a document as unclassified is a personal career bet, while stamping it classified, or just sitting on the request, is free and the latter leaves no record. The reform does exactly two things about that. It takes the free act, the “no, this is classified” away from individuals and gives it to a system that keeps receipts. And it makes the careful act, the “yes, this is unclassified” safe to perform.

Here is the reformed process. A human security officer can still say yes to openness and operational necessity at every step: approve a release, grant a downgrade, clear a manuscript, agree a paragraph needs no stamp. Nothing about that changes, and nothing new gets released by machine. What no individual can do anymore is issue the authoritative “no”. Every denial, whether of a classification challenge, a downgrade or declassification request, or a drafter’s working-level “does this need a stamp” query, comes from one accredited automated adjudication system, and the system logs each denial with the specific guide rule it applied. The person who acted on the system’s concurrence gets a good-faith safe harbor tied to that logged event. If the call later proves wrong, the liability lands on the system, which can be measured and fixed, instead of on a career, which can only be ended. And silence stops working: a query nobody answers is automatically referred to the adjudicator, within five business days for a drafter’s question and never more than 120 days for the rest. The pocket veto does not die because security officers get braver. It dies because the security office does not have to subconsciously choose between protecting themselves and national security.

Notice the exhaust this throws off. Every “no” now flows through one chokepoint that records the question, the rule, and the outcome, and participating agencies report their denial, concurrence, escalation, and measured-accuracy rates every quarter. The decision-level record this entire assessment has been starved of becomes a routine byproduct of running the system. The reform is not just a fix. It is the collection platform.

What does the AI in the middle actually do? Three narrow things, none of them a magical oracle. It retrieves the governing Security Classification Guide(s) for the program at hand. It checks the content in front of it against that guide’s rules, the same lookup a diligent human reviewer is already supposed to perform. And it returns a determination that cites the rule it relied on, under an identifier that stays with the document. When the call is close or its models disagree, it does not guess. It escalates to the cleared humans inside the same accredited system, whose decisions are logged the same way. That is the arithmetic in section 8: escalate the hard 13.78 percent and the machine’s miss rate on what it decides alone falls to 2.16 percent on my benchmark. Give the humans the hard work, that takes real judgement to adjudicate ambiguous guidance or where world events have surpassed classification guides.

None of the structure depends on the AI being good. Accreditation is the gate: a trial against held-out, independently adjudicated documents, with guide retrieval certified as part of the trial, because my benchmark showed a model handed the wrong guide is worse than one handed nothing. Until an adjudicator passes that trial, the AI-only denier stays what section 8 says it is: a hypothesis with a published test attached.

Where would it live? My bias first: I know Department of War security far better than Intelligence Community security these days. You put the adjudicator on the systems that already store the classification guides and host security’s daily workflows. On the secret network that is the Defense Technical Information Center (DTIC), which holds most of the collateral SCGs. Above it, that is the Joint Access Database Environment (JADE), which is the front-end many of security’s existing classified workflows, and shows up in security-officer job listings as a daily tool. Put the adjudicator where the rules and the officers already are, and nobody builds a new server that holds the crown jewels.

The legal path is narrower than an enthusiast would like. Executive Order reaches roughly 90 to 95 percent of the decision mass, because the current allocation of denial authority exists only by executive order and what one order allocates a successor can reallocate. It cannot touch nuclear Restricted Data, cannot extinguish criminal exposure under the espionage statutes without legislation, and cannot displace the intelligence agencies’ independent statutory withholding authorities. Moderate confidence, pending the OGC review the memo itself demands.

The collection plan

Every judgment in this assessment that sits below high confidence comes with the data collection that would resolve it, and all of it is cheap for the one party that holds the data.

How to read it: each low or moderate confidence judgment, mapped to the audit or trial that would settle it. What to see: every soft spot in this assessment is one audit away from hard, in either direction.

A representative decision-event audit with independent adjudication settles the rate, the ratchet, and the pocket-veto prevalence. A published accreditation trial on a held-out corpus of adjudicated documents settles the AI question. Deidentified sanction counts by error direction settle the asymmetry’s magnitude. A protected practitioner survey settles the chill. An institution that exercises coercive power through secrecy and declines to produce any of these measurements has made a choice, and the choice is itself informative.

Best arguments against this

The strongest counterargument: the whole cost case leans on a rate nobody measured, so I could be describing a system that over-classifies a little and calling it a crisis. I concede the premise about measurement and reject the conclusion. The direction is measured six independent ways. The floor is proven. The referee record, the elicitation, and the incentive census all point the same way, and the reform’s core is worth doing at the floor. If you believe the true rate sits at 3 percent, you are betting that every official who ever went on the record, and the appeals panel’s twenty-two-year docket, are all wrong in the same direction.

The second strongest: contested appeals are cherry-picked, so the 75 percent overturn rate overstates the problem. Partly true, and I keep the selection caveat welded to the number. But selection cuts both ways: the documents nobody can afford to appeal, inside CAPS and SAPS few ever see, face less scrutiny than the appealed ones, not more. If someone gets rejected for access to a SAP, they’re stuck on security’s slow approval process thereafter. If you’re not senate confirmed, then there are career consequences for rocking security’s boat.

The third: the AI cannot do the job. Correct, today, and this assessment says so at high confidence. That is why the reform’s core is accuracy-independent and the AI-only denier is a trial, not a deployment. The reality is that there is already a trust schism between the security apparatus, and the rest of the community. I expect security to distrust AI tools. That’s good, because the system collects data, make it better than what we have today. I hope to see the broader workforce and security band together to thread against the AI who is the bad robot saying “no.” That’s a healthy culture.

And a fourth, which deserves the most respect: some secrets are worth everything their protection costs, and a system tuned to never lose them will over-protect everything else as the price of the tail. I concede more than half of this. There is probably a line somewhere around the unacknowledged or waived access programs where the default is “no”. That’s a SAPCO call. What right-sizing cannot explain is zero sanctions, a correction channel nobody uses, secrecy orders on patents no longer get lifted even after the inventions ceases to be novel, and an enterprise that stopped measuring itself. A well-calibrated system would show a working feedback loop somewhere. This one shows none.

What would change my mind

A representative, independently adjudicated audit finding the true excess share near the 3 to 8 percent floor would collapse my cost estimate to the bottom of the slope, and I would publish that number with relief, because it would mean the measurement finally exists.

A single documented sanction for over-classification, in any channel, would bound the asymmetry and improve the paper. I searched for one and want to be sent one.

An accreditation trial in which a deployable adjudicator clears 98.6% with certified classified guide retrieval would move the AI-only denier from hypothesis to result.

A released internal audit showing agencies already track decision-level correctness with independent adjudication would retire the internal-measurement judgment outright.

The close

I went in believing over-classification was about people protecting a paycheck. It is not. It is a bet where one side ends careers and the other side has never once, in forty-seven years of public records, cost anyone anything, taken by rational people tens of millions of times a year.

Classification is not a judgment about a document. It is a bet about a career.

The intelligence community writes assessments about adversaries who hide their embarrassing and national secrets. The classification system now qualifies. Move the no somewhere it can be seen, and the government gets, for the first time, the one thing this assessment could not: the measurement of the secrets it keeps from itself.