SYNTHESIS NOTE
Topics›Agent Harness›this note

Can explicit behavior maps help weaker planners compete with stronger models?

Explores whether organizing harness repositories around runtime behavior—rather than relying on model inference—can narrow the capability gap between weaker and stronger planning models, and whether this reduces computational overhead.

Synthesis note · 2026-07-17 · sourced from Agent Harness

If Why is finding distributed behavior code so hard?, the fix is to build the missing map explicitly rather than make an agent recover it every time. Harness Handbook is a behavior-centric representation: it links behavioral descriptions to the distributed source that implements them, constructs that mapping directly from source, and supports editing through behavior-guided progressive disclosure plus automatic resynchronization when the code changes.

The results argue that the mapping — not raw model strength — was doing the work. With the handbook, overall win rates rise 10.0 and 18.9 points on Codex and Terminus-2 while planner token use falls 12.7% and 8.6%: better plans for less compute, because the planner no longer burns tokens rediscovering where behavior lives. The sharper evidence is substitutive: with the handbook a weaker planner matches the implementation-site localization of substantially stronger models, improving every file- and symbol-level Recall/Precision/F1 comparison against two independent reference plans, and the gains persist across request types and difficulty levels.

This is a concrete instance of a recurring pattern in the harness literature — capability relocating out of the model and into the scaffolding around it. Like Can externalized bookkeeping let smaller search agents beat larger ones?, the move is to externalize a structure (here, the behavior→code map) so the model spends its budget on judgment rather than bookkeeping. It also complements What happens to code that agents create and then share?: a durable, resynchronized representation is exactly the kind of persistent artifact that compounds.

Inquiring lines that read this note 46

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

How does harness optimization generalize across different model architectures and domains? How do pretraining biases affect reward signal effectiveness in RLVR? What reasoning architectures enable models to solve complex problems efficiently? How does the generation-verification gap limit what we can measure about AI reasoning? How should agent systems validate and persist generated code artifacts? What training dynamics and scale trigger emergence of reasoning capabilities? What capability trade-offs arise from domain specialization through fine-tuning? Can local safety checks guarantee system-level behavioral safety? How do capability benchmark scores systematically misrepresent true model abilities? Why do stronger reasoning capabilities create tradeoffs with instruction following? How effectively can language models perform reasoning, especially combined with symbolic methods? How do agent-learned skills transfer and improve across different tasks? Why do standard benchmarks fail to predict agent deployment success? Should agents decouple planning from perception grounding for better performance?

Related concepts in this collection 3

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
14 direct connections · 72 in 2-hop network ·medium cluster Open in graph ↗

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Original note title

reorganizing a harness repository around runtime behavior lets a weaker planner match a stronger model's code localization