How do search and reasoning workflows improve forecasting performance over base models?
This explores why wrapping a base model in retrieval, decomposition, and search loops improves forecasting — and whether the gains come from better reasoning or from better architecture around the model.
This explores why wrapping a base model in retrieval, decomposition, and search machinery beats prompting it directly — and the corpus has a clear, slightly surprising answer: the workflow, not the raw model, is doing most of the work. The most direct evidence is that LLMs turn out to be much stronger forecasters than people assumed, but only when the workflow separates numerical reasoning from contextual reasoning Can LLMs actually forecast time series better than we think?. Ask a single model to extrapolate a trend and interpret world events in one pass and you get mush; split those jobs apart and a latent capability surfaces. The Nexus system makes this concrete by decomposing forecasting into staged steps — contextualization, a dual-resolution macro/micro outlook, and synthesis — and beats both pure time-series models and plain LLMs on real data Can decomposing forecasting into stages unlock numerical and contextual reasoning?.
Search and retrieval add the other half: forecasting the future requires information the model never saw in training. A retrieval-augmented system reached near-parity with competitive human forecasters on questions published after its training cutoff — sometimes beating the crowd — without any forecasting-specific tuning Can retrieval-augmented language models forecast like human experts?. And the act of searching behaves like a tunable compute dial in its own right: deep research agents improve with more search steps along a curve that mirrors the reasoning-token scaling law, complete with the same diminishing returns Do search steps follow the same scaling rules as reasoning tokens?. So 'search' isn't just plumbing — it's a genuine inference-compute axis sitting alongside reasoning.
Here's the thing you might not expect: piling on more reasoning is a weaker lever than it sounds. On numerical and constraint-bound tasks, reasoning models don't systematically beat standard ones — extended chain-of-thought produces more text, not more actual iterative computation Do reasoning models actually beat standard models on optimization?, and models plateau around 55–60% constraint satisfaction regardless of size or training Do larger language models solve constrained optimization better?. Worse, reasoning chains fail through disorganization: models wander into invalid paths and abandon promising ones too early, an 'underthinking' problem you can partly fix with a decoding penalty rather than a smarter model Why do reasoning models abandon promising solution paths? Do reasoning models switch between ideas too frequently?. This is exactly why workflow structure matters for forecasting — the gains come from giving the model scaffolding that keeps it on-task, not from letting it think longer.
When reasoning does help, it's because of training, not inference budget. Reasoning models keep their edge no matter how much compute non-reasoning models are given, because training instills a protocol that makes the extra tokens productive Can non-reasoning models catch up with more compute?. The same lesson shows up in search agents: a 20B model that externalizes its bookkeeping to a stateful harness outperformed the next-best open searcher by a wide margin, and the gain survived ablation — the harness was a learned capability, not mere implementation Can externalizing bookkeeping improve search agent performance?.
The deeper takeaway is that 'improve forecasting over the base model' is mostly a selection-and-orchestration problem. Routing queries to the right specialist beats scaling a single model — ten small models with routing surpassed GPT-4.1 Can routing beat building one better model? — and giving recursive reasoners stochastic latent transitions lets them hold genuine uncertainty and carry multiple candidate futures instead of collapsing to one guess Can stochastic latent reasoning help models explore multiple solutions?. Forecasting is inherently a distribution-over-futures task, so the workflows that win are the ones that decompose the problem, fetch what the model can't know, and preserve uncertainty — rather than the ones that simply make the model think harder.
Sources 12 notes
LLMs have stronger intrinsic forecasting ability than recognized, but only when workflows separate numerical reasoning from contextual reasoning. Monolithic prompting obscures this capability; structured decomposition surfaces it.
Nexus outperforms pure TSFM and LLM baselines on real-world datasets by decomposing forecasting into contextualization, dual-resolution macro/micro outlook, and synthesis stages. Separating numerical extrapolation from event-driven contextual reasoning avoids forcing one model to handle both simultaneously.
A retrieval-augmented LM system achieved near-parity with competitive human forecasters on real forecasting questions published after model training cutoffs, sometimes surpassing human crowds. Newer model generations naturally improved forecasting without domain-specific tuning.
Deep research agents improve with more search steps in a pattern mirroring the reasoning-token relationship, with both exhibiting diminishing returns. This reveals a new inference-compute axis beyond model capability alone.
Reasoning variants with extended CoT show no consistent advantage over standard models on constraint-bound numerical tasks like optimal power flow. Extended thinking produces more text, not more iterative computation, suggesting the bottleneck is numeric procedure rather than reasoning steps.
Across constrained-optimization tasks, LLMs converge to ~55–60% constraint satisfaction independent of architecture, parameter count, or training regime. Reasoning models do not systematically outperform standard models, suggesting a fundamental ceiling rather than a scaling gap.
Reasoning LLMs exhibit two reinforcing failures: wandering (invalid exploration) and underthinking (premature path-switching). Decoding-level interventions like thought-switching penalties improve accuracy without fine-tuning, suggesting viable solutions exist but are abandoned prematurely.
o1-like models frequently abandon reasoning paths mid-exploration, wasting tokens on incomplete approaches. A decoding-only penalty on thought-transition tokens (TIP strategy) discourages switching, improving accuracy on challenging math without model fine-tuning.
Reasoning models persistently outperform non-reasoning models regardless of inference budget because training instills a reasoning protocol that makes additional tokens productive. The gap is fundamentally about deployment mechanisms and training structure, not raw capability.
A 20B model using Harness-1 achieved 0.730 average curated recall across eight benchmarks, outperforming the next open searcher by 11.4 points. The gains transfer to held-out benchmarks and survive ablation, showing the harness is not mere implementation but a learned capability.
Avengers-Pro achieves 7% higher accuracy than GPT-5-medium by routing queries to optimal models per semantic cluster, or matches its performance at 27% lower cost. Ten 7B models with routing previously surpassed GPT-4.1 and 4.5, suggesting selection is a stronger lever than scaling.
GRAM replaces deterministic latent updates with stochastic sampling, enabling models to represent distributions over solutions rather than single predictions. This allows handling of ambiguous problems and multiple valid strategies that deterministic designs cannot represent.