Test-Time Compute

Source papers 39

The Arxiv papers behind this sub-topic. Links may take you off-site to arxiv.org.

• A Survey on LLM Inference-Time Self-Improvement
  Techniques that enhance inference through increased computation at test-time have recently gained attention. In this survey, we investigate the current state of LLM Inference-Time Self- Improvement fr…
• A Survey on Post-training of Large Language Models
  The emergence of Large Language Models (LLMs) has fundamentally transformed natural language processing, making them indispensable across domains ranging from conversational systems to scientific expl…
• A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?
  2 What to Scale “What to scale” refers to the specific form of TTS that is expanded or adjusted to enhance an LLM’s performance during inference. When applying TTS , researchers typically choose a sp…
• Advancing Language Model Reasoning through Reinforcement Learning and Inference Scaling
  Large language models (LLMs) have demonstrated remarkable capabilities in complex reasoning tasks. However, existing approaches mainly rely on imitation learning and struggle to achieve effective test…
• Agent-R: Training Language Model Agents to Reflect via Iterative Self-Training
  Abstract: Large Language Models (LLMs) agents are increasingly pivotal for addressing complex tasks in interactive and agentic environments. Existing work primarily focuses on enhancing performance th…
• Agentic Systems as Boosting Weak Reasoning Models
  Can a committee of weak reasoning-model calls reach the performance of much stronger models? We study verifier-backed committee search as inference-time boosting for reasoning language models. The mec…
• Behavioral Exploration: Learning to Explore via In-Context Adaptation
  While humans are able to achieve such fast online exploration and adaptation, often acquiring new information and skills in only a handful of interactions, existing algorithmic approaches tend to rely…
• Deep Think with Confidence
  Large Language Models (LLMs) have shown great potential in reasoning tasks through test-time scaling methods like self-consistency with majority voting. However, this approach often leads to diminishi…
• Does Thinking More always Help? Understanding Test-Time Scaling in Reasoning Models
  This raises a natural question: Does thinking more at test-time truly lead to better reasoning? To answer this question, we perform a detailed empirical study across models and benchmarks, which revea…
• Don't "Overthink" Passage Reranking: Is Reasoning Truly Necessary?
  With the growing success of reasoning models across complex natural language tasks, researchers in the Information Retrieval (IR) community have begun exploring how similar reasoning capabilities can …
• Dynamic Rewarding with Prompt Optimization Enables Tuning-free Self-Alignment of Language Models
  Aligning Large Language Models (LLMs) traditionally relies on costly training and human preference annotations. Self-alignment seeks to reduce these expenses by enabling models to align themselves. To…
• Emergence of Abstractions: Concept Encoding and Decoding Mechanism for In-Context Learning in Transformers
  Humans distill complex experiences into fundamental abstractions that enable rapid learning and adaptation. Similarly, autoregressive transformers exhibit adaptive learning through in-context learning…
• End-to-End Test-Time Training for Long Context
  On the other hand, Transformers with self-attention still struggle to efficiently process long context equivalent to years of human experience, in part because they are designed for nearly lossless re…
• Enhancing LLM Reasoning via Critique Models with Test-Time and Training-Time Supervision
  Training large language models (LLMs) to spend more time thinking and reflection before responding is crucial for effectively solving complex reasoning tasks in fields such as science, coding, and mat…
• From Decoding to Meta-Generation: Inference-time Algorithms for Large Language Models
  One of the most striking findings in modern research on large language models (LLMs) is that scaling up compute during training leads to better results. However, less attention has been given to the b…
• Generative Recursive Reasoning
  How should future neural reasoning systems implement extended computation? Recursive Reasoning Models (RRMs) offer a promising alternative to autoregressive sequence extension by performing iterative …
• LIMOPro: Reasoning Refinement for Efficient and Effective Test-time Scaling
  Large language models (LLMs) have demonstrated remarkable reasoning capabilities through test-time scaling approaches, particularly when fine-tuned with chain-of-thought (CoT) data distilled from more…
• LLM Post-Training: A Deep Dive into Reasoning Large Language Models
  Some policy gradient approaches are explained below: Policy Gradient (REINFORCE). The REINFORCE algorithm [114, 115] is a method used to improve decision-making by adjusting the model’s strategy (poli…
• LLMs can implicitly learn from mistakes in-context
  Learning from mistakes is a fundamental feature of human intelligence. Previous work has shown that Large Language Models (LLMs) can also learn from incorrect answers when provided with a comprehensiv…
• Long-context LLMs Struggle with Long In-context Learning
  Large Language Models (LLMs) have made significant strides in handling long sequences. Some models like Gemini could even to be capable of dealing with millions of tokens. However, their performance e…
• MatFormer: Nested Transformer for Elastic Inference
  Foundation models are applied in a broad spectrum of settings with different inference constraints, from massive multi-accelerator clusters to resource-constrained standalone mobile devices. However, …
• Memorization and Knowledge Injection in Gated LLMs
  Large Language Models (LLMs) currently struggle to sequentially add new memories and integrate new knowledge. These limitations contrast with the human ability to continuously learn from new experienc…
• Meta-Reasoner: Dynamic Guidance for Optimized Inference-time Reasoning in Large Language Models
  To address these issues, we introduce Meta- Reasoner, a framework that dynamically optimizes inference-time reasoning by enabling LLMs to “think about how to think.” Drawing inspiration from human met…
• Reflect then Learn: Active Prompting for Information Extraction Guided by Introspective Confusion
  Large Language Models (LLMs) show remarkable potential for few-shot information extraction (IE), yet their performance is highly sensitive to the choice of in-context examples. Conventional selection …
• Revisiting the Test-Time Scaling of o1-like Models: Do they Truly Possess Test-Time Scaling Capabilities?
  The advent of test-time scaling in large language models (LLMs), exemplified by OpenAI’s o1 series, has advanced reasoning capabilities by scaling computational resource allocation during inference. W…
• Satori: Reinforcement Learning with Chain-of-Action-Thought Enhances LLM Reasoning via Autoregressive Search
  Large language models (LLMs) have demonstrated remarkable reasoning capabilities across diverse domains. Recent studies have shown that increasing test-time computation enhances LLMs’ reasoning capabi…
• Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters
  Enabling LLMs to improve their outputs by using more test-time computation is a critical step towards building generally self-improving agents that can operate on open-ended natural language. In this …
• Scaling Laws for Agent Harnesses via Effective Feedback Compute
  Agent harnesses increasingly determine the performance of language-model systems by deciding how models call tools, receive feedback, verify intermediate states, store memory, and revise solutions. Ye…
• Sleep-time Compute: Beyond Inference Scaling at Test-time
  Scaling test-time compute has emerged as a key ingredient for enabling large language models (LLMs) to solve difficult problems, but comes with high latency and inference cost. We introduce sleep-time…
• TTRL: Test-Time Reinforcement Learning
  This paper investigates Reinforcement Learning (RL) on data without explicit labels for reasoning tasks in Large Language Models (LLMs). The core challenge of the problem is reward estimation during i…
• Test-Time Scaling with Reflective Generative Model
  We introduce our first reflective generative model MetaStone-S1, which obtains OpenAI o3- mini’s performance via the new Reflective Generative Form. The new form focuses on highquality reasoning traje…
• The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models
  This paper aims to overcome a major obstacle in scaling reinforcement learning (RL) for reasoning with large language models (LLMs), namely the collapse of policy entropy. Such phenomenon is consisten…
• The Surprising Effectiveness of Test-Time Training for Abstract Reasoning
  Language models have shown impressive performance on tasks within their training distribution, but often struggle with novel problems requiring complex reasoning. We investigate the effectiveness of t…
• Think Deep, Think Fast: Investigating Efficiency of Verifier-free Inference-time-scaling Methods
  This work conducts a comprehensive analysis of inference-time scaling methods for both reasoning and non-reasoning models on challenging reasoning tasks. Specifically, we focus our research on verifie…
• Think Twice: Enhancing LLM Reasoning by Scaling Multi-round Test-time Thinking
  Recent advances in large language models (LLMs), such as OpenAI-o1 and DeepSeek-R1, have demonstrated the effectiveness of test-time scaling, where extended reasoning processes substantially enhance m…
• TiMoE: Time-Aware Mixture of Language Experts
  Large language models (LLMs) are typically trained on fixed snapshots of the web, which means that their knowledge becomes stale and their predictions risk temporal leakage: relying on information tha…
• Towards Large Reasoning Models: A Survey on Scaling LLM Reasoning Capabilities
  Researchers have moved beyond simple autoregressive token generation by introducing the concept of “thought”—a sequence of tokens representing intermediate steps in the reasoning process. This innovat…
• Vector Policy Optimization: Training for Diversity Improves Test-Time Search
  Language models must now generalize out of the box to novel environments and work inside inference-scaling search procedures, such as AlphaEvolve, that select rollouts with a variety of task-specific …
• interwhen: A Generalizable Framework for Steering Reasoning Models with Test-time Verification
  Reasoning models produce long traces of intermediate decisions and tool calls, making test-time verification increasingly important for ensuring correctness. Existing approaches either verify only the…