In April 2026, Microsoft announced something easy to miss in the noise of the agent gold rush. It’s the Azure SRE Agent that handled 35,000 production incidents across the company’s internal services, which saved more than 20,000 engineering hours. Microsoft built the system on a publicly available LLM, the same model family that thousands of other teams used, often with inconsistent results.
That same quarter, research on enterprise AI adoption was still circulating, showing that 95% of generative AI pilots at large companies failed to deliver measurable returns. A separate enterprise survey shared that roughly 40% of multi-agent pilots never make it past the six-month mark, and about 88% of agent prototypes never reach production.
Build Reliable Enterprise Multi-Agent Systems
The size of that gap makes the next finding uncomfortable. In a March 2026 study, researchers ran a controlled experiment in which they kept the model constant and changed only the surrounding code. They measured performance on agentic benchmarks and found a sixfold difference. Same model, same weights, just different software around it. A smaller, cheaper model with well-engineered scaffolding outperformed larger frontier models running with minimal setups. A search procedure for math tasks discovered scaffolding that also transferred across five previously unseen models.
Two conclusions follow. First, the model is not the bottleneck most enterprises assume. Second, the engineering that bridges the gap between an impressive demo and a production-ready system is neither prompt engineering nor a better framework. Over the past year, it has gained a name, emerging as the missing layer between AI research and reliable enterprise software.
It is called harness engineering. This piece introduces it: what it is, how it differs from the prompt and context engineering that came before it, why multi-agent systems make it essential, what a harness engineer actually does, and how enterprises can start building this capability without first inflating their cloud costs.
What a harness actually is
The word harness is borrowed, unmodified, from software testing. A test harness is the surrounding apparatus that sets up a system under test, runs inputs through it, captures outputs, and decides whether the result passes or fails. It is not the code being tested. It is everything around the code that makes testing reliable and repeatable.
The term has migrated into agent engineering with little semantic drift. In this context, a harness is everything around the model: the tools it can use, the memory it can access, the logic that determines what happens next, the checks that verify work before it is finalized, the rules that govern access, and the observability that helps people understand what happened and why. The model is a probabilistic function that turns inputs into outputs. The harness is the deterministic software that turns those outputs into reliable behavior.
The analogy that has stuck most usefully is this: the model is the engine, the harness is the car. Anyone who has driven a car with a powerful engine and bad brakes understands intuitively why the second one matters more in production.
The findings make the case quantitatively. Harness choices created a performance gap of up to six times: how context was assembled before each call, how tools were named and described, how errors were caught and retried, how state was carried between turns, and how outputs were validated before action. None of it lived inside the model.
The implication for the people writing cloud cheques is awkward but useful. Spending more on a frontier model is a linear lever; performance scales slowly with cost. Investing in a better harness is a non-linear lever. Same dollars, different multipliers. Most enterprises have been pulling the linear lever and wondering why their agents still feel brittle.
The three layers: prompt, context, harness
The clearest way to think about the discipline is as a three-layer stack. Prompt engineering, context engineering, and harness engineering are often discussed as alternatives or as successor disciplines, with each new term implying that the last one is dead. Neither framing is correct. They are nested.
Figure 1. Prompt, context, and harness engineering as nested layers around the model.
Prompt engineering operates at the message level. It governs how teams send the actual text to the model in a single turn: instructions, examples, output format, role framing, and refusal conditions. It is the smallest of the three disciplines, but it has not gone away. A bad prompt is still one of the fastest ways to ruin an otherwise sound system. It is simply no longer the dominant lever.
Context engineering operates at the context-window level. It decides what information the model is allowed to see at each step: which documents are retrieved, which conversation history is kept verbatim, which is summarised, which tool outputs are kept and which are dropped, which memory is loaded, and from where. Most production failures that look like hallucinations are not hallucinations; they are context failures. The model reasoned correctly over the wrong information.
Harness engineering operates at the system level. It determines how the model is invoked at all: how the agent loop runs, which tools the agent can access, what validation runs on its outputs before any action is taken, what happens when something fails, how the policy is enforced, and how the whole system is monitored, evaluated, and improved. The harness is where deterministic software meets probabilistic intelligence.
The relationship between the three layers is straightforward once it is named. The harness assembles the context. The context contains the prompt. The prompt instructs the model. Each layer assumes the one below it is in place. Polishing the prompt while the context is wrong is overclocking a CPU on a machine with no swap space. Stacking more agents while the harness is brittle is like adding cores to a system with no scheduler.
The table below summarises the differences along the dimensions that actually matter when an enterprise is staffing the work.
| Dimension | Prompt Engineering | Context Engineering | Harness Engineering |
| What it controls | Instructions to the model | Information available to the model | The system around the model |
| Scope | A single message | A single inference call | The entire agent runtime |
| Emerged around | 2022–2023 | 2024–2025 | 2025–2026 |
| Failure it prevents | Misinterpretation, format errors | Hallucination, missing knowledge | Unreliable behavior at scale, unsafe actions |
| Primary artefact | The prompt text | The retrieval and summarisation pipeline | The orchestration runtime, evals, and policies |
| Typical owner | Anyone touching the model | ML or data engineers | Platform/agent engineers |
| Tooling needed | None – text only | RAG, vector stores, memory stores | Orchestration frameworks, evals, observability, sandboxes |
| Question it answers | What should the model think? | What should the model see? | What should the model be allowed to do? |
Table 1. Comparison of the three disciplines along practical, staffing-relevant dimensions.
Most teams that struggle in production have been answering the wrong question. They tune the prompt for hours when the failure is in retrieval. They invest in retrieval when the failure is a missing retry, a missing approval gate, or an absent verifier. The diagnostic question is which row of the table the failure lives on. Once that is named, the fix usually becomes obvious.
Anatomy of a harness
Past the metaphors, a harness is a set of components. Frameworks differ in how they package them. LangGraph relies on graph-based control flow, the OpenAI Agents SDK relies on handoffs, and the Claude Agent SDK relies on a managed loop. The components themselves are remarkably stable. Eight of them recur in nearly every production system worth studying.
Figure 2. The eight components of an agent harness, with the observability layer running alongside everything else.
A short tour, in roughly the order things happen at runtime.
Policy and permissions are the first gate. Before any model call is made, the harness establishes who is asking, what scopes they have, and what the agent is allowed to do on their behalf. In a serious enterprise system, this is not a system-prompt instruction. It is not a control to instruct the model to “not delete production data”. It is a structured permission system enforced outside the model. Anthropic and others have written at length about why “beyond permission prompts” is non-negotiable for autonomous work.
Context assembly is where context engineering lives, hosted inside the harness. The harness decides which retrieval to run, which memory to load, which conversation history to keep verbatim, and how to summarise the rest. The Microsoft team that built Azure SRE Agent eventually moved from a prompt-driven design with over a hundred bespoke tools to a filesystem-based context system that gave the model a structured workspace. The change was a harness change, not a prompt change.
Memory and state is the durable substrate. It includes the working set the model can see right now, the longer-term store of past interactions, and the handoff artifacts that let an agent resume work after a context reset: feature lists, progress files, and structured task plans. Anthropic’s long-running coding harness, which builds full applications across multi-hour autonomous runs, depends on these handoff files. Without them, every new context window is amnesia.
The orchestration loop is the heart of the harness. It runs the observe-think-act cycle, decides when to call the model and when to call a tool, manages turn budgets, and triggers compaction when the context begins to bloat. Most production failures attributed to “the agent” are actually orchestration-loop bugs.
The tool interface is where the model meets the rest of the world. Tool design is genuinely under-appreciated as a discipline. Names, descriptions, parameter schemas, and error messages collectively determine whether the model can use a tool effectively. Anthropic’s engineering team has been blunt about this: tool design is agent UX. A harness with 40 tools and ambiguous names will almost never outperform a harness with 8 well-named, well-scoped tools.
Verification is the most underbuilt component in failing systems. After the model produces an output (a code change, a JSON document, or a proposed action), a separate process should verify the output before it is acted on. The verifier may be deterministic (schema validation, type checks, dry-run execution), another model with no write access, or human approval. Anthropic’s three-agent harness for long-running coding pairs a generator with an evaluator that runs in a fresh context window with no write tools, on the principle that the same agent should never both produce and grade its own work.
Error recovery is the harness’s response to the inevitable. Tools time out. Models refuse. Outputs fail validation. A serious harness defines retry budgets, fallback strategies, replanning conditions, and clear escalation paths to humans. The naive single-retry-with-the-same-prompt pattern is a significant source of production costs.
Observability and evaluation sit alongside everything else and never sleep. Without traces of every tool call, every model output, every state transition, debugging multi-step systems is a guessing game. Without an evaluation harness, a fixed set of inputs and outputs that the system is regularly graded against, there is no way to tell whether a change made things better or worse. The teams that get this right treat evals less like tests and more like a continuously running benchmark.
These eight components are the spine of a harness. The question for an enterprise is not whether to build them. That decision is made implicitly the moment an agent goes live. The question is whether they are built with intent or assembled by accident.
Conclusion
Harness engineering distinguishes AI systems offering measurable operational value from mere impressive demos. As models become more capable, the focus shifts from the model itself to the supporting engineering systems. Reliable AI behavior emerges from a cohesive system that includes orchestration, memory, verification, and policy enforcement, rather than prompts alone. In multi-agent environments, structured coordination and production-grade engineering are essential to address architectural challenges.
Successful organizations will prioritize building strong operational frameworks around AI, recognizing agents as integrated components of business workflows. Harness engineering transforms probabilistic models into trustworthy systems and becomes foundational as multi-agent architectures shift from experimentation to production.
In Part 2, we’ll examine why multi-agent systems make harness engineering indispensable, how production architectures evolve, and what enterprises must build to scale reliable AI systems.
