TOPIC
Model Routers
A subject the collection covers, read through 7 synthesis notes.
View as
How quickly does behavioral diversity plateau in model pools?
When you combine multiple language models into a routing system, does adding more models keep improving behavioral coverage, or do you hit diminishing returns fast? This matters for designing efficient multi-model systems.
Multi-agent scaling follows three quantitative laws with measurable thresholds Small language models suffice for most agentic subtasks cost-effectively Routing requires behavioral differentiation and paraphrase stability Multi-agent performance is primarily determined by token spending. Semantic capability vectors enable scalable agent discovery without manual wiring.
Can routing queries to task-matched structures improve RAG reasoning?
Does matching retrieval structure type to task demands—tables for analysis, graphs for inference, algorithms for planning—improve reasoning accuracy over uniform chunk retrieval? This explores whether cognitive fit principles from human learning transfer to AI systems.
When do graph databases outperform vector embeddings for retrieval? Can reasoning topologies be formally classified as graph types? Can organizing knowledge structures beat raw training data volume? Can query-time graph construction replace pre-built knowledge graphs? How should retrieval and reasoning integrate in RAG systems?
Graph databases outperform embeddings for relational domain queries requiring multi-hop traversal Reasoning topologies classify formally as path graphs, trees, and directed graphs Knowledge structure outperforms volume in domain model training Query-time logic graphs eliminate pre-built graph costs and inflexibility Retrieval and reasoning integrate most effectively through MDP formulation and process-level supervision.
What makes query complexity a better routing signal than response quality? Does sentence-level granularity capture enough structure for complex reasoning tasks? What graph structures better support multi-hop reasoning than pairwise edges? Can retrieval improve multi-step reasoning by triggering at each uncertainty? Why does visual similarity retrieval fail for embodied agents? Can task-aware ranking replace similarity scoring in other RAG systems? How should we allocate compute between reasoning and retrieval iterations? Does parallel retrieval outperform sequential search chains at test time? How can per-step decisions about knowledge retrieval improve reasoning over uniform policies? What makes proactive tool retrieval better than single-round semantic matching? What replaces truth-correspondence in probabilistic knowledge representations? How much does organized knowledge improve learning efficiency versus raw data?See all 122 inquiring lines on this note →
Can five components unify all LLM routing approaches?
Does decomposing LLM routers into context encoders, model encoders, scoring functions, decision rules, and learning signals create a fair comparison framework across single-turn, multi-turn, and personalized routing methods?
LLM routing selects models before generation by estimating query complexity Multi-agent routing requires optimizing four decisions at once Routing queries to specialized models outperforms frontier models Routing requires behavioral differentiation and paraphrase stability Small language models suffice for most agentic subtasks cost-effectively
Does model selection matter more than model improvement for query routing? What learning signals best supervise router training across benchmark tasks? How do cost-aware cascades compare to single-turn routing in the component framework? Who serves as the teacher model in the routing-guided distillation process? How does hierarchical routing in the concept module influence token-level generation?
Can routers select the right model before generation happens?
Explores whether LLMs can be matched to queries by estimating difficulty upfront, before any generation begins. This matters because routing could cut costs significantly while preserving response quality.
Adaptive inference compute allocation outperforms fixed budgets per prompt Smaller models with more inference compute match larger ones on hard prompts Small language models suffice for most agentic subtasks cost-effectively Routing queries to specialized models outperforms frontier models LLM routers decompose into five comparable components
Why does single-model routing beat ensemble and cascade approaches on latency? How do routing and test-time compute scaling work together as optimization axes? What makes query complexity a better routing signal than response quality? Can routing enable heterogeneous SLM-first architectures at scale? Should model routing decisions account for prompt-tier dependencies? Can this distillation pattern apply beyond e-commerce to other latency-constrained domains? Why is latency budget a constraint for e-commerce rankers? Can model routing and compute allocation work together as independent optimizations? How do byte-level models allocate compute without explicit difficulty estimators? What constraints force mobile deployments to operate in the sub-billion parameter regime? Can routing systems prevent expert models from failing outside their specialty? Does the optimal model size depend on what capabilities you actually need?See all 53 inquiring lines on this note →
What decisions must multi-agent routing systems optimize simultaneously?
Standard LLM routing only picks which model to use. But multi-agent systems involve four interdependent choices: topology, agent count, role assignment, and per-agent model selection. Does optimizing all four together actually improve performance?
Can AI systems design unique multi-agent workflows per individual query? Can multi-agent teams automatically remove their weakest members? When does adding more agents actually help systems? Can routers select the right model before generation happens? Can semantic capability vectors replace manual agent routing?
Query-level meta-agents generate personalized multi-agent systems via RL feedback Contribution scoring enables inference-time deactivation of low-performing agents Multi-agent scaling follows three quantitative laws with measurable thresholds LLM routing selects models before generation by estimating query complexity Semantic capability vectors enable scalable agent discovery without manual wiring.
Should model routing decisions account for prompt-tier dependencies? Can model routing and compute allocation work together as independent optimizations? What specific network sizes trigger coordination degradation in LLM systems? How should topology routing adapt to different task types? What structural constraints does topology impose on role and LLM assignment? Can construction-time routing and runtime agent pruning be combined effectively? How do multi-agent routers balance flexibility against interpretability in design? How do routers decide when to escalate from small to large models? Can multiple small models outperform a single large model with good routing? How does routing decide between models before generation happens? What four decisions matter most in multi-agent system routing? When and what should a model actually decide to delegate?See all 15 inquiring lines on this note →
When does routing between models actually matter?
Routing systems are typically evaluated on accuracy alone, but this misses whether the models are truly specialized and whether routing decisions remain consistent across paraphrased queries. What structural conditions make routing non-vacuous?
LLM routing selects models before generation by estimating query complexity Routing queries to specialized models outperforms frontier models Multi-agent routing requires optimizing four decisions at once Model routing and execution control are themselves attack surfaces. Behavioral diversity in model pools saturates within fewer than ten agents
Can routing policies remain meaningful over behaviorally homogeneous model pools? How do KNN and prompted routers differ in the accuracy-stability tradeoff? Can a router predict query complexity well but still fail as governance? What makes routing a governance mechanism rather than just a predictor? How much safety burden shifts between provider filters and model alignment in rerouted requests? Does model selection matter more than model improvement for query routing? Does the improved model actually return to the routing pool and shape future decisions? Why does a second routing level sometimes break accuracy in disclosure hierarchies?
Can routing beat building one better model?
Does directing queries to specialized models via semantic clustering outperform investing in a single frontier model? This challenges whether model improvement or model selection drives performance gains.
Adaptive inference compute allocation outperforms fixed budgets per prompt Smaller models with more inference compute match larger ones on hard prompts Small language models suffice for most agentic subtasks cost-effectively LLM routing selects models before generation by estimating query complexity Model routing and execution control are themselves attack surfaces.
Why do naive baselines outperform trained models in entity-level CRS evaluation? Why does single-model routing beat ensemble and cascade approaches on latency? What makes query complexity a better routing signal than response quality? Can routing enable heterogeneous SLM-first architectures at scale? Should model routing decisions account for prompt-tier dependencies? Can this distillation pattern apply beyond e-commerce to other latency-constrained domains? Why is latency budget a constraint for e-commerce rankers? How does nesting optimization levels improve on traditional network depth? Can semantic clustering of stakeholders preserve meaningful evaluative diversity without manual curation? Can bilevel autoresearch discover new search mechanisms for the inner research loop? Can bilevel autoresearch succeed when the inner and outer loops use different models? Can model routing and compute allocation work together as independent optimizations?See all 80 inquiring lines on this note →