Can better tools fix LLM document editing errors?
Does giving LLMs agentic tool access—like diffing, re-reading, or structured editors—improve their reliability on long-horizon document workflows? Understanding whether the problem is tool limitations or decision-making quality matters for reliability engineering.
A natural intuition for fixing LLM document corruption: give the model better tools. Let it diff its own output, re-read the file, call a structured editor instead of regenerating prose. The DELEGATE-52 evaluation tests this directly and finds that agentic tool access does not improve performance on the benchmark.
The finding rules out a class of proposed fixes. Tool wrappers, ReAct loops, and structured editing affordances are not addressing the failure mechanism — they are downstream of it. The degradation comes from the model's own decisions about what to change and how, not from limitations of the editing interface. A model that decides to flip a numeric value will flip it through any tool you give it.
This also disambiguates two senses of "agent." The first sense — LLM-plus-tools, where capability is gated on tool affordances — predicts that tool access should improve document workflows. The second sense — LLM-as-decider, where the model's judgment about what to edit is the bottleneck — predicts that tool access should be roughly orthogonal. The DELEGATE-52 result favors the second.
The implication for workflow design: reliability gains on long delegated work probably come from changing what the model decides (better prompting, verification loops, decomposition into smaller reversible steps) rather than from upgrading what it can act through. Tool engineering helps when capability is interface-limited; it does not help when capability is judgment-limited.
Inquiring lines that use this note as a source 17
This note is a source for these synthesized inquiries. Follow a line forward into its question, or open it to trace back to all of its sources.
- What repair strategies work best at each level of Clark's ladder?
- Do architectural changes or training fixes better prevent agreement failures?
- How do insert-expansions and third position repair together cover full repair lifecycle?
- What makes draft-centric systems better anchors for coherence than feed-forward outputs?
- Is prompt engineering a workaround rather than a capability fix?
- How do insert-expansions differ from third position repair in timing?
- What makes provenance infrastructure more critical than artifact quality?
- How should organizations redesign workflows if LLMs cannot solve optimization directly?
- Why do frontier models corrupt more documents than weaker models during workflows?
- What causes silent document corruption in long LLM workflows?
- Where does agent reliability come from if not better tools?
- Why do LLMs choose incorrect edits despite understanding the task?
- What detection mechanisms work best for corruption-style document errors?
- Why do frontier model failures in document editing go undetected by users?
- How does model tier affect whether errors delete or corrupt document content?
- Can tool use or self-conditioning fix degradation in extended LLM workflows?
- What unique perspective do designers bring to LLM adaptation that engineers might miss?
Related concepts in this collection 3
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Do frontier LLMs silently corrupt documents in long workflows?
Explores whether advanced language models introduce undetectable errors when delegated multi-step tasks, and whether degradation continues accumulating beyond initial rounds of processing.
same paper, the parent finding
-
Why do language models fail to act on their own reasoning?
LLMs produce correct explanations far more often than they produce correct actions. What causes this knowing-doing gap, and can training methods close it?
adjacent: knowledge does not transfer to action
-
Where does agent reliability actually come from?
Exploring whether LLM agent performance depends on larger models or on thoughtful system design choices like memory, skills, and protocols that shift cognitive work outside the model.
partial counterpoint: harnessing helps elsewhere
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- LLMs Corrupt Your Documents When You Delegate
- Towards a Science of Scaling Agent Systems
- Problems with Cosine as a Measure of Embedding Similarity for High Frequency Words
- Solving a Million-Step LLM Task with Zero Errors
- interwhen: A Generalizable Framework for Steering Reasoning Models with Test-time Verification
- Why Do Multi-agent LLM Systems Fail?
- Agents of Chaos
- Recommender AI Agent: Integrating Large Language Models for Interactive Recommendations
Original note title
agentic tool use does not improve llm document-editing reliability — tools are not a fix for long-horizon delegation drift