How does query planning as a separate step improve multi-hop retrieval coherence?
This explores whether splitting retrieval into a distinct planning stage — figuring out what to look for before fetching and answering — keeps multi-hop reasoning from drifting off course across steps.
This explores whether splitting retrieval into a distinct planning stage — deciding what to look for before fetching and synthesizing — produces more coherent multi-hop answers than a single tangled pass. The corpus says yes, and the clearest reason is architectural: when you fold query planning into answer synthesis, the two interfere with each other. Separating them into their own components reduces that interference and measurably improves performance on multi-hop queries Do hierarchical retrieval architectures outperform flat ones on complex queries?. The same separation-of-concerns principle shows up in agent design generally, where keeping planning apart from execution is a documented win — so this isn't a RAG quirk, it's a recurring structural pattern.
What's interesting is that the corpus frames planning less as 'write a better query' and more as 'choose the right shape for the problem.' StructRAG routes each query to a knowledge structure that fits its demands — a table, a graph, an algorithm, a catalogue, or plain chunks — using a trained router, and grounds this in cognitive-fit theory: reasoning improves when the representation matches the task Can routing queries to task-matched structures improve RAG reasoning?. That's planning as a first-class decision. The deeper motivation is that retrieval failures are structural, not things you tune away: fixed retrieval intervals waste context, embeddings measure association rather than relevance, and there are hard mathematical limits on what a vector can represent Where do retrieval systems fail and why?. A planning step is one of the few levers that addresses the architecture rather than the dials.
There's a productive tension worth seeing. One camp turns retrieval into an explicit, staged process you can scale: CoRAG extends chain-of-thought training to retrieval, generating intermediate retrieval chains and letting you spend more test-time compute — greedy for speed, tree search for accuracy — exactly like reasoning-token scaling Can retrieval be extended into multi-step chains like reasoning?. The opposing camp tries to collapse the multi-hop dance into a single shot: HippoRAG builds the corpus into a knowledge graph and runs Personalized PageRank from the query's concepts to traverse multiple hops at once, matching iterative retrieval at a fraction of the cost and latency Can knowledge graphs enable multi-hop reasoning in one retrieval step?. So 'planning as a separate step' and 'plan away the steps entirely' are two routes to the same coherence goal — explicit staging versus a structure that encodes the hops in advance.
Coherence also lives in how evidence accumulates between hops, not just in how the query is planned. HGMem stores retrieved evidence as hyperedges so three or more entities can bind into a single relation without being chopped into pairwise links — which keeps joint constraints intact as reasoning compounds across steps Can hypergraphs capture multi-hop reasoning better than graphs?. And planning has a budget cost: long-horizon research agents degrade when unrestricted reasoning inside one search turn eats the context needed for later rounds, so capping per-turn reasoning — not just total time — preserves the agent's ability to fold in new evidence Does limiting reasoning per turn improve multi-turn search quality?.
The surprise at the edges: a separate planning step may sometimes be solving a problem you could dissolve elsewhere. Uncertainty estimation — just reading the model's own calibrated token probabilities to decide when to retrieve — beats elaborate multi-call adaptive retrieval on single-hop and matches it on multi-hop, far more cheaply Can simple uncertainty estimates beat complex adaptive retrieval?. And fine-tuning the retriever itself can absorb work you'd otherwise hand to a planning stage: a model trained on implicit queries matches augmented retrievers without expanding the input or adding an explicit augmentation step Can fine-tuning replace query augmentation for retrieval?. So the honest answer is that planning-as-separation reliably helps coherence — but whether you want it as an explicit stage, baked into a graph, or trained into the retriever is the real design question the corpus is circling.
Sources 9 notes
Separating query planning from answer synthesis into distinct components reduces interference and improves multi-hop query performance. This architectural principle mirrors documented benefits of separating planning from execution in agent design.
StructRAG demonstrates that selecting knowledge structure type based on query demands—via DPO-trained router choosing among tables, graphs, algorithms, catalogues, and chunks—improves knowledge-intensive reasoning over standard retrieval. The approach grounds this in cognitive load and cognitive fit theory from cognitive science.
RAG systems fail at three structural levels: adaptive triggering (fixed intervals waste context), semantic-task mismatch (embeddings measure association, not relevance), and mathematical limits (embedding dimension constrains representable document sets). These require fundamentally different retrieval approaches, not tuning.
CoRAG extends chain-of-thought training to retrieval by using rejection sampling to generate intermediate retrieval chains. Test-time compute can scale through chain length and count, creating a compute dial—greedy decoding for speed or tree search for accuracy—just like reasoning-token scaling.
HippoRAG converts corpus into a knowledge graph, then uses Personalized PageRank seeded from query concepts to traverse multi-hop paths in one step. It matches iterative retrieval while being 10-20x cheaper and 6-13x faster, with 20% better accuracy on multi-hop QA.
HGMem organizes retrieved evidence as hyperedges rather than flat lists or binary graphs, allowing three or more entities to bind into single relations without decomposition. This structure accumulates coherent knowledge across retrieval steps, trading representational complexity for constraint expressiveness.
Unrestricted reasoning within single search turns consumes context needed for subsequent retrieval rounds, degrading the agent's ability to incorporate new evidence. Setting per-turn reasoning budgets, not just overall time limits, prevents this context erosion and maintains search quality across iterations.
Calibrated token-probability uncertainty consistently beats multi-call adaptive retrieval on single-hop tasks and matches performance on multi-hop, using a fraction of the LM and retriever calls. The model's self-knowledge proves more reliable than external heuristics for deciding when to retrieve.
Fine-tuned semantic search models trained on implicit queries match the performance of augmented pretrained retrievers without expanding input length. The model learns to resolve ambiguity through training rather than requiring explicit augmentation.