When should you optimize agent behavior versus tool performance separately?
This explores when it pays to treat an agent's decision-making (planning, memory, coordination) and the tools it uses as two separate optimization problems — and when fixing one won't fix the other.
This explores when it pays to treat an agent's decision-making and its tools as separate optimization targets. The corpus has a surprisingly clean answer: separate them whenever the two live on different cost curves — because the research keeps finding that agent efficiency doesn't move as one lump. One note frames this directly: agent efficiency decomposes into three orthogonal axes — memory compression, tool learning, and planning — each with its own cost profile in tokens, latency, and steps, and improving one does *not* automatically improve the others Does agent efficiency really break down into three distinct components?. Orthogonality is the whole reason to optimize separately: if tool performance and agent behavior were coupled, one knob would do. They aren't, so they don't.
The sharpest case for separation comes from the work that physically decouples a trainable curator from a frozen executor. When you let a separate trained component evolve the *skill library* while the agent that runs those skills stays fixed, the repository shifts from generic verbose additions toward actionable, reusable strategy — and the trained curator even generalizes across different executor backbones Can a separate trained curator improve skill libraries better than frozen agents?. That's the pattern in miniature: improve the tool/skill layer on its own clock, and the behavior layer inherits the gains without retraining. The same logic shows up in the idea that reliable agents externalize memory, skills, and protocols into a harness layer rather than asking the model to re-solve them every run — the harness is precisely where you optimize tooling independently of the reasoning Where does agent reliability actually come from?.
But the corpus also marks where separation breaks down — where tool behavior and agent behavior are entangled enough that you must co-design. Dynamic tool discovery during execution beats pre-retrieved tool sets on long-horizon tasks, which means the 'tool layer' isn't a static thing you can tune offline; the agent's runtime strategy and which tools exist are decided together Can agents discover tools dynamically instead of pre-selecting them?. Similarly, the production finding that direct function calls beat protocol-mediated tool access — and that single-tool-per-agent design restores determinism — is really about tool *interface* shaping agent *reliability*: a tooling decision that you cannot evaluate without watching agent behavior Why do protocol-based tool integrations fail in production workflows?. And API-first interaction cutting task time 65–70% over UI clicking shows the tool surface itself can dominate end-to-end performance, independent of how clever the agent's planning is Can API-first agents outperform UI-based agent interaction?.
The practical rule the corpus implies: separate the optimization when the layers sit on different cost curves and one can be improved offline — skill curation, memory compression, swapping small models in for cheap repetitive subtasks while reserving large models for the hard calls Can small language models handle most agent tasks?. Co-optimize when the tool's existence or interface is itself a runtime strategic decision. There's a unifying frame here worth knowing: language agents can be written as computational graphs where node prompts (behavior) and edge connectivity (coordination/tool flow) are two distinct axes you can optimize automatically and independently Can we automatically optimize both prompts and agent coordination?. That formalizes the intuition — sometimes you tune the nodes, sometimes the wiring.
One last thing that reframes the whole question: a lot of what looks like 'better agent behavior' is just spending more tokens. Research attributes ~80% of multi-agent performance variance to token budget rather than coordination intelligence How does test-time scaling work at the agent level?, and interaction-time scaling (more environment steps) turns out to be orthogonal to chain-of-thought reasoning depth Does agent interaction time scale separately from reasoning depth?. So before you decide whether to optimize behavior or tools, the corpus suggests checking whether you're actually optimizing either one — or just paying for more steps. And measure accordingly: single-score task success hides exactly the separation you're trying to manage, which is why agent evaluation is pushing toward trajectory quality, memory hygiene, and verification cost as distinct dimensions What should we actually measure in agent evaluation?.
Sources 11 notes
Research identifies memory compression, tool learning efficiency, and planning optimization as three structurally independent components, each with distinct cost profiles (tokens, latency, and steps). Improving one axis does not automatically improve the others, requiring holistic design.
SkillOS shows that separating a trainable curator from a frozen executor, grouped by task streams, causes skill repositories to shift from generic verbose additions toward actionable execution logic and cross-task meta-strategies. The trained curator generalizes across different executor backbones and domains.
Research shows reliable LLM agents externalize three cognitive burdens—memory (state persistence), skills (procedural components), and protocols (structured interaction)—into a harness layer rather than relying on model scale alone. The harness unifies these externalities and eliminates the need for the model to solve the same problems repeatedly.
DeepAgent demonstrates that discovering tools as needed—rather than pre-retrieving a fixed set—enables agents to maintain global task perspective and adapt strategy mid-execution. This approach scales better for long-horizon tasks where the tool space is too large to enumerate.
MCP integration caused non-deterministic failures through ambiguous tool selection and parameter inference. Replacing it with explicit direct function calls and single-tool-per-agent design restored determinism. A 306-practitioner survey confirms 85% of production teams build custom agents, forgoing frameworks.
The AXIS framework shows that prioritizing API calls over sequential UI interactions cuts task completion time by 65–70% while maintaining 97–98% accuracy and reducing cognitive workload by 38–53%. A self-exploration mechanism automatically discovers and constructs APIs from existing applications, solving the bootstrapping problem.
SLMs handle the repetitive, well-defined language tasks that constitute most agent work at 10–30× lower cost than LLMs, making heterogeneous architectures (SLMs by default, LLMs selective) the economically rational design pattern.
Language agents represented as computational graphs—where nodes are operations and edges define information flow—reveal that CoT, ToT, and Reflexion are formally equivalent structures. This unified view enables automatic optimization of both node prompts and edge connectivity without manual redesign.
Research shows 80% of multi-agent performance variance comes from token budget, not coordination intelligence. LatentMAS and shared-KV-cache approaches offer ways to decouple performance gains from token costs.
Test-time interaction—increasing environment steps—enables exploration, backtracking, and replanning that per-step reasoning cannot achieve. Curriculum-based RL on rollout length produces SOTA web agents, showing interaction scaling dominates on tasks with partial observability.
Single-score evaluation collapses multi-dimensional agent behavior and creates false confidence in deployment readiness. Research shows agents need benchmarks for trajectory quality, memory hygiene, context efficiency, and verification cost to reflect actual system performance.