Can question quality be trained separately from the decision to ask?
This explores whether two distinct skills — asking a *good* question (clarity, relevance, specificity) and deciding *whether to ask at all* — can be trained as separate capabilities, and what the corpus says about how each is learned.
This reads the question as two skills hiding inside one phrase: knowing *how* to phrase a useful question, and knowing *when* a question is worth asking instead of guessing. The corpus treats these as genuinely separable problems, trained by different machinery — and that separation turns out to be the interesting part.
On the quality side, the work suggests question quality isn't one thing but a decomposable space. The ALFA framework breaks "good question" into theory-grounded attributes — clarity, relevance, specificity — and trains on attribute-specific preference pairs, beating models trained on a single quality score Can models learn to ask genuinely useful clarifying questions?. That mirrors findings elsewhere: argument quality can't be absorbed from labeled examples alone, because models just learn surface patterns unless given an explicit framework like RATIO Can models learn argument quality from labeled examples alone?, and prompt quality similarly resolves into six measurable dimensions grounded in Gricean and cognitive-load theory rather than a flat checklist Can we measure prompt quality independent of model outputs?. The throughline: quality is a structured, supervisable target you can optimize directly.
The *decision to ask*, by contrast, is a behavioral disposition that shows up nowhere on a quality rubric — and it's where standard training actively works against you. CollabLLM shows that next-turn reward optimization trains models to respond passively rather than probe for intent, because immediate helpfulness is what gets rewarded Why do language models respond passively instead of asking clarifying questions?. The same dynamic surfaces as an "alignment tax": preference optimization rewards confident answers over clarifying questions, cutting grounding behaviors 77.5% below human levels Does preference optimization harm conversational understanding?. So the *willingness* to ask has to be installed by a different mechanism — RL that rewards proactive critical thinking (raising it from near-zero to 74% on flawed problems) Can models learn to ask clarifying questions instead of guessing?, or social meta-learning that makes a model treat conversation itself as an information source and delay answering when underspecified Can models learn to ask clarifying questions without explicit training?, Can LLMs learn to ask for feedback during problem solving?.
So the answer is yes — and the two channels can even move in opposite directions. The SFT "accuracy trap" is the warning shot: fine-tuning can raise final-answer correctness while degrading the actual reasoning quality by 39%, because the metric you optimize and the capability you care about come apart Does supervised fine-tuning improve reasoning or just answers?. Read against the question, that means you could plausibly train a model to ask beautifully phrased questions it never decides to ask, or to interrupt constantly with poorly-formed ones — each axis optimized blind to the other.
The quietly useful corollary the corpus surfaces: deciding *when* to ask is really a question-*selection* problem, and there's machinery for that too. Active-learning approaches pick the single most informative question to reduce uncertainty — ten adaptive questions can pin down a user's preferences Can user preferences be learned from just ten questions? — and question *type* itself determines the right downstream strategy Does question type determine the right retrieval strategy?. Which hints that the cleanest framing isn't "quality vs. decision" but a pipeline: decide whether to ask, decide which question maximizes information, then phrase it well — three trainable stages, not one.
Sources 11 notes
The ALFA framework breaks down question quality into theory-grounded attributes (clarity, relevance, specificity) and trains models on 80K attribute-specific preference pairs. Attribute-specific optimization outperforms single-score training, especially in clinical reasoning where asking the right clarifying question directly impacts decision quality.
Fine-tuning on labeled examples fails to transfer quality criteria to new argument types. Models learn surface patterns rather than principled criteria. Explicit instruction using frameworks like RATIO or QOAM significantly improves performance and generalization.
Research identifies six evaluable dimensions—Communication, Cognition, Instruction, Logic, Hallucination, and Responsibility—with 20 sub-criteria based on Grice, cognitive load theory, and instructional design. Improvements in one dimension cascade to others, revealing prompt quality as a structured space rather than a flat checklist.
CollabLLM demonstrates that standard RLHF training optimizes for immediate helpfulness, discouraging models from asking clarifying questions or offering multi-turn insights. Multi-turn-aware rewards that estimate long-term interaction value enable active intent discovery and genuine collaboration.
RLHF optimizes models for single-turn helpfulness by rewarding confident responses over clarifying questions and understanding checks. This preference alignment systematically reduces grounding acts by 77.5% below human levels, creating an alignment tax where models appear helpful but fail silently in multi-turn contexts.
Reinforcement learning training increased proactive critical thinking accuracy from 0.15% to 73.98% on deliberately flawed math problems. Notably, inference-time scaling degraded this ability in untrained models but improved it after RL training, suggesting the capability is learnable but fragile without explicit training.
Models trained via SML on complete problems generalize to underspecified tasks by asking for needed information and delaying answers. The training paradigm instills a meta-strategy of using conversation as an information source, addressing the premature-answering failure mode.
Research shows that reformulating static tasks as pedagogical dialogues—where a teacher has privileged information and the student must learn to extract it—trains models to actively engage conversation as a problem-solving tool, not just imitate dialogue patterns.
Supervised fine-tuning improves final-answer accuracy on benchmarks but cuts Information Gain by 38.9 percent, meaning models generate correct answers through post-hoc rationalization rather than genuine inferential steps. Standard metrics miss this degradation because they only measure final correctness.
PReF learns base reward functions from preference data, then uses active learning to select maximally informative questions that reduce coefficient uncertainty. Users can be personalized via inference-time reward alignment without weight modification.
Research shows non-factoid questions split into five types, each requiring different retrieval and aggregation methods. Evidence-based questions suit standard RAG, while debate and comparison need aspect-specific retrieval, and experience/reason questions need decomposition or filtering strategies.