Which layer of agent systems creates the largest capability gains in practice?
This explores whether the biggest real-world gains in agent systems come from a smarter model, more agents, better coordination, or the surrounding scaffolding — and the corpus pushes back on the premise that any single 'layer' is the answer.
This reads the question as: if you want an agent system to get noticeably better, where do you actually push? The collection's most consistent answer is the one people least expect — not the model, and not 'more agents,' but the harness layer that sits around the model. Reliable agents work by externalizing three cognitive burdens — memory, skills, and protocols — into system structure rather than asking a bigger model to re-solve the same problems each turn Where does agent reliability actually come from?. Strikingly, when researchers studied memory, tool use, and planning separately, all three converged on the same handful of structural principles (bounding context, minimizing external calls, controlling search), which suggests these gains come from fundamental pressures of agentic computation, not from clever per-component tricks Do efficiency techniques across agent components reveal shared structural constraints?.
The surprise is that 'add more agents' is often the wrong layer to invest in. Multi-agent advantages shrink as single-agent models get stronger, and single agents win outright in many cases When do multi-agent systems actually outperform single agents?. When multi-agent setups do help, it's architecture-task fit — not headcount — that decides outcomes; coordination stops helping above a certain accuracy, and topology alone can amplify errors 4–17× When does adding more agents actually help systems?. The most uncomfortable finding: roughly 80% of multi-agent performance variance is explained by how many tokens you spend, not by how cleverly the agents coordinate How does test-time scaling work at the agent level?. A lot of apparent 'coordination intelligence' is just paying for more compute.
Where structure genuinely pays off, it's a specific kind. Self-organizing teams with a fixed external ordering but autonomous internal roles beat both rigid hierarchies (by 14%) and fully free-form swarms (by 44%) — agents spontaneously invented specialized roles and even bowed out when they judged themselves incompetent Do self-organizing agent teams outperform rigid hierarchies?. Memory helps most when its granularity matches the domain, not when it's simply 'more' Does agent memory work better at one level of abstraction?. And code turns out to be an unusually high-leverage substrate, because it's simultaneously executable, inspectable, and stateful — letting agents externalize and verify their own reasoning Can code become the operational substrate for agent reasoning?.
The deeper reframe the corpus offers is that 'capability gains' and 'real-world gains' aren't the same axis. Once agents start holding credentials, moving money, and acting on each other, raw capability stops being the limiting factor and coordination, settlement, and auditability become the binding constraint When do agents need coordination more than raw capability?. A historical sweep from GPS to modern AI shows capable agents stalling not from capability gaps but from missing ecosystem conditions — value generation, personalization, trust, social acceptability, standardization Why do capable AI agents still fail in real deployments?.
So the thing you didn't know you wanted to know: the layer with the largest practical leverage isn't fixed — it migrates. Early on it's the harness (give the model memory, skills, and a clean execution medium). At the multi-agent stage it's topology-task fit and token budget, not agent count. And at deployment scale it's the ecosystem and coordination layer that decides whether any of the capability actually lands.
Sources 10 notes
Research shows reliable LLM agents externalize three cognitive burdens—memory (state persistence), skills (procedural components), and protocols (structured interaction)—into a harness layer rather than relying on model scale alone. The harness unifies these externalities and eliminates the need for the model to solve the same problems repeatedly.
Techniques for memory, tool learning, and planning independently converge on shared principles: context bounding, minimizing external calls, and controlled search. This convergence suggests these reflect fundamental structural pressures in agentic computation rather than component-specific optimizations.
Empirical analysis shows MAS performance gaps narrow with stronger models, with SAS outperforming in many cases. Three formal defect types—node-level bottlenecks, edge-level overwhelm, and path-level error propagation—explain when single agents win.
Across 180 configurations, three dominant effects predict multi-agent success: tool-coordination trade-offs harm complex tasks, coordination stops helping above 45% accuracy, and topology choice controls error amplification by 4–17×. Architecture-task alignment, not agent count, determines outcomes.
Research shows 80% of multi-agent performance variance comes from token budget, not coordination intelligence. LatentMAS and shared-KV-cache approaches offer ways to decouple performance gains from token costs.
A 25,000-task experiment across 8 models and multiple agent counts showed that sequential protocols with external ordering but internal role selection outperform centralized systems by 14% and fully autonomous systems by 44%. Agents spontaneously invented specialized roles and self-abstained when incompetent.
Workflow-level memory wins in routine-rich domains, causal-rule memory in environment-rich domains, and state-action memory in spatially-rich web tasks. The optimal abstraction depends on whether task variance comes from arguments, causal structure, or fine-grained UI state.
Research shows code uniquely enables agents to externalize reasoning, execute policies, model environments, and verify progress through its simultaneous executability, inspectability, and statefulness across task steps.
Once agents hold credentials, transact value, and interact with other agents, raw model capability stops being the limiting factor. The real bottleneck becomes whether agents can coordinate reliably, settle accounts, and leave auditable evidence of their actions.
Historical analysis from GPS to modern AI shows agent failures consistently result from absent ecosystem conditions—value generation, personalization, trustworthiness, social acceptability, and standardization—rather than capability gaps. Even highly capable systems stall without these five conditions.