Can a proposer agent actively surface a solver's weaknesses to prevent plateau?
This explores whether a 'proposer' agent — one that generates challenges or probes — can deliberately target a 'solver' agent's blind spots to keep it improving instead of stalling out at a performance plateau.
This explores whether a proposer agent can deliberately target a solver's blind spots to keep it improving. The corpus doesn't have a paper named for the proposer-solver setup directly, but it has assembled the pieces that explain *why* this dynamic works — and what makes it fail. The core insight comes from the limits of static training: agents trained on fixed expert demonstrations are capped by 'curator imagination,' unable to learn from their own failures because they never face challenges calibrated to where they're actually weak Can agents learn beyond what their training data shows?. A proposer is, in essence, a way to replace that frozen curriculum with a live one that adapts to the solver. The clearest evidence that adaptive, empirical pressure beats plateau is the Darwin Gödel Machine, which abandons fixed proofs for trial-and-error against benchmarks and keeps an evolving archive of variants — getting 2.5× on SWE-bench precisely because the challenge environment keeps moving Can AI systems improve themselves through trial and error?.
Sources 8 notes
Agents trained on static expert datasets cannot learn from their own failures or generalize beyond demonstrated scenarios because they never interact with environments during training. Competence is capped by what curators imagined, not by agent capacity.
DGM replaces formal proofs with empirical benchmarking and maintains an evolutionary archive of agent variants, achieving 2.5× improvement on SWE-bench and 2.2× on Polyglot by discovering capabilities like better code editing and context management.
Reliability for long-trace reasoning comes from checking intermediate states and policy compliance during generation, not from scoring final outputs. Adding intermediate verification raised task success from 32% to 87% because most failures are process violations, not wrong answers.
Reasoning LLMs exhibit two reinforcing failures: wandering (invalid exploration) and underthinking (premature path-switching). Decoding-level interventions like thought-switching penalties improve accuracy without fine-tuning, suggesting viable solutions exist but are abandoned prematurely.
RLAD jointly trains abstraction and solution generators, showing that allocating test-time compute to diverse abstractions outperforms parallel solution sampling at large budgets. Abstractions create structured breadth-first exploration that prevents the underthinking failure mode of depth-only reasoning chains.
Research shows LLMs including ChatGPT cannot initiate topics, plan strategically, or lead conversations because their training optimizes for responding to queries, not creating dialogue from agent goals. This passivity is reinforced by alignment objectives and masked by fluent-sounding outputs.
Research shows next-turn reward optimization structurally removes initiative from models, but proactive behaviors like critical thinking and clarification-seeking are trainable (0.15% to 73.98% with RL). The core challenge is balancing proactivity with civility to avoid intrusion.
RLVMR uses structured meta-reasoning tags (planning, exploration, reflection, monitoring) with programmatic rewards to train agentic RL. This reduces repetitive actions by 31% compared to outcome-only methods while maintaining better generalization than supervised fine-tuning alone.