The plumbing that connects an AI to real actions is harder to edit than the AI itself — one behavior hides in a dozen places.
What makes harnesses more tangled than other types of agent code?
This explores why the 'harness' — the infrastructure code that connects a model's outputs to real actions — is harder to reason about and edit than the model or the artifacts it produces at runtime.
This explores why the harness — the infrastructure layer that wires a model's outputs to tools, prompts, and staged actions — ends up more tangled than other agent code. The starting point is that agent code isn't one thing. It splits into three layers with different failure modes: the model's trained internal capability, the system-provided harness that turns outputs into actions, and the artifacts the agent writes while running What are the three distinct layers of agent code?. The harness is the awkward middle layer, and its tangle comes from a specific structural fact about how it stores behavior.
The core reason: a single *behavior* in a harness is smeared across many *locations* in the code. Prompts, tool definitions, and pipeline stages each hold a piece of any given behavior, so there's a mismatch between how you'd describe a change ('make it stop retrying dead endpoints') and where that behavior actually lives structurally. The hardest part of changing a harness isn't writing the edit — it's finding every place the behavior is implemented. That localization problem, not code generation, is the real bottleneck Why is finding distributed behavior code so hard?. Tangle here is literally distribution: no one file is the behavior.
That framing pays off when you try to fix it. If you reorganize the repository around runtime behavior — an explicit map from behaviors to the code that implements them — even a weaker planner can locate the right spots as well as a much stronger model, with double-digit win-rate gains and fewer tokens spent hunting Can explicit behavior maps help weaker planners compete with stronger models?. In other words, the tangle isn't intrinsic to the task; it's an artifact of organizing code structurally instead of behaviorally.
There's a second kind of tangle worth naming: when the harness tries to improve itself, its 'learning' is easy to overstate. Inspecting evolved harness edits shows most of them just cache task-specific fixes the agent could have rediscovered in a single run, rather than distilling reusable strategy Do harness edits learn reusable strategies or memorize task fixes?. And because harness evolution is itself a search loop, apparent design gains are confounded with raw search effort unless you compare against an equal-budget baseline on held-out tasks How should we measure gains from automatic harness evolution?. So the harness is tangled both to read (behavior is distributed) and to credit (improvement blurs with memorization and search).
The deeper lesson the corpus keeps circling: harness tangle tracks how integration is done. Production teams find that protocol-mediated tool access (like MCP) introduces non-deterministic failures through ambiguous tool selection, and that replacing it with explicit direct function calls restores predictability Why do protocol-based tool integrations fail in production workflows? — with 85% of teams building custom agents rather than adopting frameworks Should coordination protocols wrap existing systems or replace them?. The through-line: harnesses get tangled when a behavior has no single home. Whether the fix is behavior-centric repo organization or deterministic direct calls, the win comes from collapsing 'where does this behavior live?' back down to one answerable place.
Sources 7 notes
Long-running agentic systems decompose into model-internal capabilities (trained reasoning), system-provided harness (infrastructure connecting outputs to actions), and agent-initiated artifacts (code created during execution). Each layer fails and improves differently, and this separation clarifies where to intervene.
The core difficulty in evolving production harnesses is not generating edits but finding every code location that implements a behavior. Harnesses distribute single behaviors across files, functions, and stages, creating a representational mismatch between behavioral requests and structural code organization.
A behavior-to-code mapping representation improved win rates by 10–19 points while reducing planner tokens by 8–13%. Weaker planners using this mapping matched stronger models' code localization across all precision and recall metrics.
Analysis of evolved harness trajectories shows rational, well-motivated edits across prompt and tool layers, but most persist fixes an agent could rediscover in a single rollout. Gains remain limited because memorized shortcuts cache what's already within reach rather than converting hard failures into successes.
Automatic harness evolution must be compared against task-level test-time search under equal feedback and inference budgets. Only gains beyond what matched search achieves are attributable to the harness design itself, not just more computation.
Show all 7 sources
MCP integration caused non-deterministic failures through ambiguous tool selection and parameter inference. Replacing it with explicit direct function calls and single-tool-per-agent design restored determinism. A 306-practitioner survey confirms 85% of production teams build custom agents, forgoing frameworks.
Research shows that agent coordination standards achieve adoption by composing existing protocols like MCP and DIDComm under a shared substrate, rather than competing to replace them. Bridging lets value accrue incrementally without forcing ecosystem-wide rewrites.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Harness Handbook: Making Evolving Agent Harnesses Readable,Navigable, and Editable
- Rethinking the Evaluation of Harness Evolution for Agents
- Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agents
- HarnessDev: Can LLMs Create and Evolve Their Own Agent Harness?
- Ouroboros: A Self-Developing Frontier Coding Agent with Reviewed Core Evolution
- An Empirical Study of Harness Design for Coding Agents
- ModularRSI: Modular and Generalizable Recursive Harness Self-Improvement
- Code as Agent Harness