How can expensive models efficiently support cheap models in production?

This explores the division-of-labor question: not "big model vs. small model," but how the two collaborate in a live system so you pay for expensive inference only where it changes the answer. The corpus converges on a clear pattern — heterogeneous architectures where cheap models do the bulk of the work and expensive models are reserved for the few steps that genuinely need them.

The foundational claim is that most production work doesn't need a frontier model at all. Small language models handle the repetitive, well-scoped subtasks that make up the majority of agent workflows at 10–30× lower cost, which makes "small by default, large selectively" the economically rational design rather than a compromise Can small language models handle most agent tasks?. The question then becomes *how* to route the selective calls. One answer is the pre-generation router: estimate a query's difficulty before anyone generates anything, and send only the hard ones to the expensive model — RouteLLM and Hybrid-LLM get 40–50% cost cuts this way, and because it's a single-model decision rather than running both and comparing, latency stays low Can routers select the right model before generation happens?. The other answer is to split a single task across tiers: hierarchical RAG hands query reformulation, passage pruning, and citation to a cheap model like Gemini Flash and reserves the expensive model purely for final synthesis — which turns out to be both cheaper *and* better than running the big model on everything Can smaller models handle RAG filtering while larger models focus on synthesis?.

A more surprising form of "support" is the expensive model never touching production at all — it supports the cheap model offline by manufacturing its training data or verifying its outputs. But the corpus complicates the obvious intuition here. For generating diverse synthetic data, smaller models around 500M parameters actually beat larger ones per sample, because big models concentrate probability mass on their favorite outputs and lose variety Why aren't bigger models better for generating diverse outputs?. And a committee of cheap model calls can match a strong model — but only when an external soundness signal (a test, a proof, a type check) exists to pick the correct answer out of the pile; sampling alone amplifies coverage without selecting When can weak models match strong model performance?. That same lesson recurs in self-improvement research: a model can't reliably bootstrap itself, and every method that works smuggles in an external anchor — a stronger judge, a past version, tool feedback Can models reliably improve themselves without external feedback?. So the expensive model's most durable role may be as the *verifier or anchor*, not the generator.

There's also a third lever that lets a cheap model punch above its weight without any expensive model in the loop: spend more compute at inference time. On hard prompts specifically, a small model given more inference-time compute can match a much larger one — pretraining scale and inference scale trade off against each other rather than being separate resources Can inference compute replace scaling up model size?. This reframes the whole question: sometimes "support from a bigger model" is better replaced by "let the small model think longer on the hard cases the router flagged."

The quiet warning across these notes is that cheap models fail in ways averages hide. They can post identical benchmark numbers while carrying fractured internal representations that shatter under distribution shift Can models be smart without organized internal structure?, and in long-horizon agent runs their own earlier mistakes contaminate the context and trigger non-linear collapse — a failure that scaling doesn't fix but test-time "thinking" partly does Do models fail worse when their own errors fill the context?. The takeaway the corpus leaves you with: the expensive model supports the cheap one most efficiently not by doing the work, but by being the difficulty router, the offline verifier, and the safety net for exactly the cases where cheap models quietly break.