Agent harness, or everything that is not the model

For three years, the question that organised every conversation about AI at work was always the same: which model are you using? We compared models the way we compared processors twenty years ago, with tables, scores and a reasonable faith that the bigger number predicted the outcome. People switched vendors over two points on a benchmark. Entire threads argued over whether a version had quietly got worse. All the budget and all the attention pointed at the same place.

That question has stopped predicting much of anything.

Since February, the industry has had a word for what does predict it, and has adopted that word at a speed that ought to raise suspicion: harness. In six months it has gone from the internal jargon of three or four teams to the title of OpenAI engineering reports, Anthropic blog posts, arXiv papers, its own Wikipedia entry and a line item in job descriptions. The suspicion is warranted, and we will come back to it. But the shift the word describes is real, it is measurable, and it has immediate practical consequences for anyone working with agents.

A model, on its own, only turns text into text

It is worth starting with the boring part, because that is where the idea lives. A language model does exactly one thing: it takes text and returns text. It does not remember what happened five minutes ago, it cannot open a file, it cannot run the test suite or find out that it failed, and it does not know when to stop. When you watch an agent read your repository, run the tests, look at the error, fix it and try again, none of those actions is performed by the model. They are performed by the software wrapped around it.

That software is the harness: everything that is not the model. The formula that took over this year is deliberately simple — agent = model + harness. Vivek Trivedy of LangChain sealed it in March with a line that has been quoted to exhaustion ever since: if you’re not the model, you’re the harness.

That definition contains a very concrete list. The loop that calls the model over and over. The catalogue of tools it can invoke and the code that actually runs them. The sandbox they run in. The permission system that decides what happens without asking. Context management, including compaction when the window fills up. Memory across sessions. Instruction files like AGENTS.md. The tests, linters and hooks that verify the result. And the observability that lets you reconstruct, afterwards, what on earth happened.

The heart of all that fits in six lines of pseudocode: assemble the context, call the model, read what it asks for, execute it, hand back the result, repeat until it declares itself done. The loop was never the hard part. The hard part is everything you have to decide inside it: what goes into the context and what gets thrown away, which tool is exposed and which is hidden, what runs without asking permission, what counts as “finished” and who checks.

Every new word has widened the perimeter of the work

The genealogy of the term explains more than its definition does.

First came prompt engineering: the work was finding the right sentence. Then came vibe coding — Karpathy, February 2025 — which was not a technique but a declared surrender: accept the code without reading it and let the model hold the wheel. In June 2025, Tobi Lütke proposed context engineering and Karpathy amplified it a week later; what mattered was no longer the sentence but exactly what sat in the context window at each step. Anthropic formalised it in September as the discipline of curating the optimal set of tokens during inference.

And in February 2026, Mitchell Hashimoto published the account of his own adoption of these tools with an idea the industry turned into a slogan within days: every time the agent makes a mistake, change the environment so that the mistake cannot happen again. Six days later, OpenAI published a report titled, plainly, “Harness engineering”. LangChain dissected the concept in March. Thoughtworks turned it into a method in April. Hugging Face had to publish a glossary in May because nobody could agree any more on where a scaffold ended and a harness began.

The trajectory is coherent and says something uncomfortable. A sentence, a window, an entire environment: the vocabulary has been moving steadily away from the model. Each new term has widened the perimeter of what counts as engineering work on the side of the person using the tool, not the one selling it.

Twenty-three points of difference without touching the model

What separates this word from the previous ones is that this time there are numbers.

Harness-Bench, published in May, is the most serious attempt to isolate the variable: 106 realistic tasks across software engineering, data analysis and knowledge work, 5,194 execution trajectories, eight different models and six configurable harnesses. The gap between the best and the worst, with identical tasks and the same model underneath, was 23.8 percentage points. It is not an isolated case: the same Claude Opus 4.5 scores 45.9% on SWE-bench Pro under a standardised scaffold and 55.4% inside Claude Code. LangChain moved from 30th to 5th place on Terminal-Bench 2.0 without changing the model at all, purely by rewriting the environment. That is more than a generational upgrade usually buys.

The mechanism is no mystery, which is why it convinces. Almost everything that separates a failed attempt from a good one happens outside the model. If the agent can run the tests, it sees the error and corrects it; if it cannot, it announces that everything works and moves on untroubled. Anthropic has catalogued three recurring failure modes that make the point: declaring victory without verifying, context anxiety — the model starts rushing as the window fills — and trying to solve everything in a single pass. None of the three is fixed by a smarter model. All three are mitigated by external verification, which is to say, by a harness.

The extreme case was published by OpenAI in February: a three-person team, an empty repository in August 2025, five months, a million lines of production code and 1,500 merged pull requests without writing code by hand. The detail almost nobody quotes from that report is the most useful one: at first it did not work. Productivity was low because of a badly assembled environment, poor tool integration and weak error recovery. It only climbed as they built the harness, piece by piece.

One caveat from Harness-Bench itself is worth adding, because it deflates the slogan a little: stronger models show less variance across harnesses. The environment always matters, but it matters disproportionately when the model is weak. A good harness is not a substitute for a good model. It is an amplifier, and also a cushion.

Yes, it is platform engineering repainted

Now for the marketing part, which does exist.

Stuart Miller wrote it up in May with little patience: harness engineering is platform engineering painted a different colour. The list of ancestors he enumerates is hard to argue with — middleware, SRE, control plane design, service-oriented architecture, even the Toyota Production System. The word is not even new within the field: test harness and eval harness have meant roughly the same thing for decades and years respectively. The one substantive difference Miller concedes is the good one: here the central component is stochastic. It does not return the same answer twice, and you have to design an environment that recovers gracefully when that component invents an action or declares finished what plainly is not. That is a difference of degree, not of nature.

And still the name earns its keep, for an unromantic reason: it marks a contractual boundary. Saying “model” and “harness” in the same sentence is saying what you buy and what you build. That boundary is exactly where the money is now. If the environment explains more variance than the model, the environment is the product: Claude Code, Codex and Cursor do not sell weights, they sell harnesses, and the subscription pays for the loop, not the inference. On the customer side the reasoning is symmetrical and works just as well: if your rules, your validations and your permissions live in the harness, the model becomes an interchangeable part. The same word serves as a moat for one side and as insurance against lock-in for the other. That is why everyone adopted it so fast.

The harness is also the industry’s best excuse

Any word that assigns responsibility gets used to deflect it before it gets used to accept it.

“It’s not the model, it’s your harness” has been the standard reply to every degradation report since March. Sometimes it is true. But it is worth seeing the whole operation: the vendor keeps the credit for the capabilities — those belong to the model, and are advertised as such — and externalises the blame for the failures onto the environment the customer assembled. It is a very convenient asymmetry, and there is no way to audit it from outside.

There is a second consequence, more technical and worse. If swapping the harness moves twenty points, no comparison between models means anything unless the harness it was measured on is disclosed. A May paper puts it bluntly: agent scores on long-horizon tasks are not valid for cross-model comparison unless a single harness is locked for everyone, or harness choice is treated as a controlled factor and its variance reported. Practically no commercial announcement satisfies either condition. We are still reading model tables that are, in fact, configuration tables.

And there is a third effect, and it arrives on the invoice. Every control you add to the harness — tests, a second agent reviewing, semantic verification of each step — burns tokens, often more than the original task. An agent’s reliability is bought in consumption, at precisely the moment when the flat rate has stopped covering agents. It is no accident that Gartner expects over 40% of agentic projects to be cancelled before 2027: the cost of building the harness is the line item nobody budgeted for, because six months ago it did not even have a name.

A harness is built out of repeated mistakes

The practical part is simpler than the vocabulary suggests, and you can start today with what you already have.

The underlying principle is Hashimoto’s, and it works like a ratchet: no rule enters the environment without a real failure behind it. When the agent gets something wrong, do not scold it in the chat; change the environment so that specific mistake becomes impossible or detects itself. Document which failure each rule answers to. That way the harness grows by evidence rather than by superstition, which is how most instruction files end up.

Always prefer executable verification over written instruction. Birgitta Böckeler offers a genuinely useful distinction here: controls can be guides, which anticipate the action, or sensors, which observe it afterwards; and they can be computational — linters, types, tests, hooks: deterministic, cheap, instant — or inferential — another agent reviewing: expensive, slow and arguable. The operating rule is to push the computational as far left as possible, before the commit, and save the inferential for after integration. A failing test teaches more than three paragraphs of AGENTS.md, and it costs no context.

On that file, the most repeated and least followed advice: keep it short. Addy Osmani recommends under sixty lines, and his comparison is exact — it is a pilot’s checklist, not a style guide. Every line competes for attention with the actual problem. The same applies to tools: ten well-scoped ones outperform fifty overlapping ones, and exposing all of them at startup is one of the most reliable ways to degrade the context before you have begun.

Finally, separate the doer from the judge. Agents approve their own work with remarkable generosity, so the evaluation step should run in a different instance, with a different context and different criteria, or on a machine outright. And keep the trajectories: without a record of what the agent did and why, you are not engineering the environment, you are speculating about it.

You are not buying an agent, you are buying half of one

None of this requires choosing between enthusiasm for the buzzword and contempt for it. It is both a rebrand of disciplines that already existed and the most precise name we have for something that genuinely changed: where the work that decides whether any of this functions actually lives.

What the term made visible is not a new technology but a division of labour. The product you buy is deliberately incomplete, and the missing half — the one holding your rules, your tests, your permissions and your context — is the one you build, pay for in tokens, and answer for when it fails.

For three years the question was which model you were using. From this year on, the useful question is what you have built around it.