Essays on harness #1: the method before the code

My machine rebooted twice on the same day, and it took the open work down with it.

That was on 09/16/2026. The culprit was a test suite I had written myself, one that opened a headless Chrome per route of the site. There was no bug: every browser did exactly what I asked. The error was somewhere else, somewhere no unit test looks, which is the sum. Five days earlier, on 09/11, a session had opened 148 Chrome processes in parallel and occupied 15.5 GB of resident memory, and I had found that funny.

Six days before that, on 09/10, the graphics card in the same machine dropped off the PCIe bus. Xid 79. The cause wasn’t a driver or a cable: it was the ACPI _ON method timing out while trying to wake the device from the deepest power state. The card was sleeping too well.

Two accidents, neither caused by broken code. Both caused by a system doing exactly what I told it to, in a context I hadn’t measured. Everything that follows comes from there, and that’s why this essay opens with the accident and not with the method. The method is the consequence; if I started with it, I’d be telling the story backwards, which is the most comfortable and least honest way to tell it.

This is the first of a series about the harness I use to work with AI agents. Harness, here, is the literal meaning: the set of rules, gates, and measurers that decides what an agent can do, what it has to prove, and who checks it. Essay #1 is a panorama. Each axis goes only as far as planting a debt, and the debt becomes the next essay.

History: the harness was born from accident, not doctrine

There was no founding moment. There was a sequence of things that went wrong, and rules written on top of them, each one with a date and the file that originated it.

The middle row of that table is the most interesting one, and the one I like least. On 09/12/2026 I decided the personal site would not use the harness, because the harness cost too much for the size of the work. On 09/21/2026 I reversed that decision. What changed wasn’t my opinion about rigor: it was the price. The harness got cheap enough, and then it fit.

That’s the first thing I learned that I didn’t expect: rigor is a function of cost. A correct method that’s too expensive doesn’t get adopted, and a method that isn’t adopted protects no one. The 09/12 decision stayed on record as wrong instead of being erased, and that isn’t archivist’s scruple: the house rule is that an amendment gets recorded, not erased, because a history where only the right decisions survive is a history that can’t predict anything.

The contract

The harness has six rules, and they fit in one paragraph.

Three states, and only three: PASS, FAIL, UNKNOWN. UNKNOWN blocks the same as FAIL, because “I couldn’t measure it” never becomes approved, and the day it does is the day the system stops being useful. PASS requires a RED before it: a criterion can only pass if it was first seen failing, since a criterion born green isn’t acceptance, it’s a guard. The measurer is frozen by hash before implementation starts, and a hash mismatch between red and green invalidates the whole proof. Partial green isn’t green: an execution that exits with a nonzero code produces PASS on no criterion, not even the ones that passed. And an amendment gets recorded, not erased.

Six sentences. None of them mention artificial intelligence, and that’s exactly the point.

Scientific method: where the contract is falsificationism, and where it isn’t

If you read those six rules and heard an echo, the echo is real, and it’s worth naming where it comes from.

RED before GREEN is a positive control: before I trust that the instrument approves, I prove it can fail. It’s the lesson from Feynman in Cargo Cult Science (1974), the text about science that copies the form of the ritual and loses its substance, and it’s the best description I know of a test suite that only ever shows green. The UNKNOWN that blocks is Popper taken seriously by the boring route: a single genuine counter-instance is enough, and the absence of an instance is not confirmation. Freezing the measurer by hash is preregistration, the practice science adopted after discovering, with Ioannidis in Why Most Published Research Findings Are False (2005), that analytical flexibility alone produces positive findings in industrial quantity. If I can touch the measurer after seeing the result, I measured nothing.

And now the part that cost me more to write, because it’s the part where the analogy doesn’t favor me: where the contract is not scientific method.

The harness tests one hypothesis at a time against a criterion I wrote myself. That’s verification, not strong inference. What’s missing is what Platt called Strong Inference (1964): rival hypotheses and an experiment that distinguishes between them, instead of a single hypothesis that passes or doesn’t. Missing is replication by an instrument of different structure, rather than the same instrument run twice, which only repeats the same failure modes. And missing is an effect estimate that can fail by magnitude, not only by direction: today my criteria say “it improved,” almost never “it improved by this much, and below that I would have failed it.”

One accounting detail that dismantles my own defense: the research behind this section has 21 declared sources, 18 verified, and three I couldn’t open - the 1964 Platt, in a third-party mirror, An Excess of Positive Results by Scheel, Schijen and Lakens (2021), and The preregistration revolution by Nosek and colleagues (2018). They’re marked unverified in my base, and they stay marked here. Citing a source I didn’t open and not saying so would itself be a cargo cult.

The debt: essay #2 is about the crucial experiment, rival hypotheses, and why my contract still has neither.

Mathematics: the ratio that measures, and the number that refuses to become a rate

A project that promises savings needs a single number, and mine is tokens read per item delivered in PASS. Baseline frozen before any change: 1,804,802,598 tokens read for 197 approved items, across 29 plans, in a window declared as 09/18 to 09/20/2026. The division gives 9,161,434.51.

Nine million tokens per delivered item.

That number is deliberately raw, and it’s worth more for what it doesn’t measure than for what it does. It doesn’t separate cache from fresh reads, it doesn’t weight by price - a cached read costs a fraction of fresh input, and ignoring that inflates the number in a way that doesn’t correspond to money. It doesn’t know how to distinguish a hard item from a trivial one, so improving the ratio by doing easy work is trivially possible and completely useless. And its window isn’t the same as another measurement I took, of 68 sessions and 15.12 billion tokens of cache reads, which doesn’t even declare the time window it covered. Two rulers, two scopes, and I don’t know if they measure the same period. That’s written in the dossier behind this text, and written there before it was written here.

Now the part I’m proud of, and the reason this essay exists.

I keep a dispatch statistic: every piece of work that goes to an agent is logged with a label, a model, and an outcome. Today it says this:

Look at the right-hand column. The last two rows have 100% approval. Two for two. One for one. No failures. And the measurer refuses to call it a rate, because the house’s minimum sample is 5, and it prints the reason in full: n=2 < 5, sample too small, rate UNKNOWN.

A system that denies its own victory for insufficient sample size. That’s what I wrote all of this for. The temptation to look at two hits out of two and write “Opus is at 100% on taste pieces” is enormous, it’s honest in intent, and it’s statistically empty: two coin flips give two heads with probability one in four. The only defense against that temptation is a rule written before I see the result, one that doesn’t obey me afterward.

The top row, with n=12 and 92%, is the only one with enough mass to be read. And it has a real failure inside it: one FAIL out of twelve, and three retries - three times a piece came back to be redone. I’m not going to soften that. It’s the number I have.

The debt: essay #3 is about the price-weighted ratio, which is the number I should really be using and still don’t.

Neurophysiology: the most expensive organ, and two analogies I had to bury

The question that opened this part of the project was naive and productive: if a token is energy, what does the brain know about spending little that an agent harness doesn’t know yet?

I found three things, and all three had a counterpart in my consumption ledger. The brain doesn’t pay to transmit what the receiver already predicted - in predictive coding, what goes up from one level to the next is only the residual between the prediction and what happened, and the dopamine neuron stays quiet when the reward arrived as expected. In the harness, an agent’s return describes everything it did, including what whoever asked already knew. What should go up is the deviation, with magnitude. Second: the brain doesn’t keep active what isn’t in use, and the filter happens before the input, not after - the cortex never pays for what was blocked along the way. In the harness, the only real cut of a command’s output is rewriting the command before running it, because once it’s in context there’s no rule that takes it out. Third: cleanup is offline, and it happens while sleeping. My observation log had 830 open entries and zero review. A trail that doesn’t evaporate isn’t memory, it’s noise.

And now the part that matters most, in the same breath, because separating the two would be dishonest.

Two analogies I wanted to use were checked against the literature and fell.

The first is hemispheric division as a global property - the hemisphere of narrow attention versus the one of broad vigilance, which is a beautiful metaphor for leader and executor. Nielsen and colleagues, across 1,011 people and 7,266 brain regions (PLOS ONE, 2013), show that lateralization is a local property of networks, not a global one. The reading I wanted to make is interpretation, not data.

The second is the 3/4 exponent of Kleiber’s law, which I wanted as a ruler for metabolic scaling and therefore for cost scaling. Two 2026 reviews contest its universality: the 0.75 measurement appears in planarians, while small mammals and invertebrates report an exponent close to 0.9. The number doesn’t work as a ruler.

An analogy that doesn’t survive the source doesn’t enter the project, and I’d rather count the two that died than the six that survived, because the ones that died are what tells you whether the process works.

One last honesty about this section. The neurophysiology research behind it has 42 declared sources and 34 verified, and it still comes out with low confidence. The reason isn’t a weak source or lack of verification: it’s concentration. Of the 34 verified, 18 - 53% - live on the same domain, the open mirror of biomedical literature. They’re articles from different journals, but the gate counts the host, and a gate that counts the host is right: if that one mirror is biased, I’d have no way to know. The distrust here is about source concentration, not source quality, and those two things get confused all the time.

The debt: essay #4 is about the brain’s three rules turned into gates, and about the project’s only quality measurement - comparing sessions that sleep against sessions that don’t.

Inspirations and references

The ideas came from four places, and none of them is software engineering.

From the philosophy of science, Popper’s falsificationism and Platt’s strong inference, plus Feynman as an uncomfortable conscience. From the replication crisis, Ioannidis and the practice of preregistration, which is where the hash freeze comes from. From systems biology, the idea that a system with many agents and a small budget was already optimized by natural selection long before I showed up. And from the neurophysiology of cost, the notion that energy economy isn’t peripheral optimization: it’s the central design pattern.

Little came from engineering, and what little did is old: Dijkstra in On the reliability of programs (1970), with the observation that testing shows the presence of a defect and never its absence, and Andreas Zeller’s material on systematic debugging.

The accounting of sources, because it too is part of the method: the four pieces of research behind this series add up to 146 declared sources, of which 99 are external and 126 were verified - meaning opened and checked, not cited from memory. The ones that weren’t are marked unverified in the base and stay marked when they appear in text, like the three in this edition. Model memory doesn’t count as a source, and a claim without a source is UNKNOWN.

Problems solved

What’s already been proven in execution, not just in intent:

• The three-state contract works, and it’s checked by machine, not by reading.
• The hash freeze caught at least one case where the measurer would have been adjusted after I saw the result.
• The accidents became gates. The three September crashes are rules that refuse the command, rather than warnings that arrive afterward.
• The small-sample signal refused two 100% rates that I would have published without blinking.
• The dispatch statistic exists, with 12 records in the busiest label. It’s small, but it’s small and measured, which is different from nothing.

Problems still open

Here the honesty has to be greater, because this is the section where it’s easiest to be vague.

When I run the map’s verifier against my own state, today, it answers: pass 53, fail 11, unreachable 17. And, below that, a line I wrote on a day I was feeling brave: FAIL is process regression: the map claims something that stopped being true.

Eleven fronts failing against myself. There’s no way around this, because the verifier refuses the entire map as long as one front claims something that’s no longer true.

That’s by far the most valuable thing I’ve built: not a system that approves my work, but one that won’t let me pretend.

It’s worth separating the states, because open and blocked are not the same thing. Open is work that can start now; blocked is work that depends on something else happening first, and starting it would be waste.

About the last row, a number I like and that proves nothing yet: the local model running on this machine’s card does 27.04 tokens per second with an optimal context of 8,192 tokens. It’s fast enough to be tempting and short enough to be useless for most pieces. Whether it’s useful or not, the measurer will say - not me.

What’s coming in the series

Every axis in this essay left a debt, and the debt is the index:

• #2, the crucial experiment. Rival hypotheses, replication by a different instrument, and why verification isn’t strong inference.
• #3, the weighted ratio. The number I should be using, why the raw one deceives, and what happens when two rulers don’t cover the same window.
• #4, the brain as a specification. The three rules turned into gates, and the project’s only quality measurement.
• #5, what the analogy doesn’t prove. Where biology gives a hypothesis with a mechanism and where it becomes decoration.

The thesis that runs through all of them: a verification system that only knows how to approve verifies nothing. Mine says 11 failures today. That’s why I still believe in it.

The numbers in this essay come from an evidence dossier frozen before the writing, and every one of them has a named origin in a measured file. The ones that didn’t were left out of the text.