Can we predict when a specific prompt will fail on a given question?
This explores whether failure is predictable in advance — given a particular prompt and a particular question, can we tell beforehand that the pairing will produce a wrong answer, rather than only discovering it after the output disappoints?
This explores whether failure is predictable in advance — whether we can flag a prompt-question pairing as doomed before running it. The corpus says: surprisingly often, yes, and from several independent angles. The most direct claim comes from treating the model as an autoregressive probability machine: tasks whose correct answer is a low-probability sequence are systematically harder, even when they're logically trivial. That framing correctly predicted failures like reciting the alphabet backwards or counting letters — before any experiment was run Can we predict where language models will fail?. So one predictor is structural: how improbable is the target answer in the model's training distribution?
A second predictor is the model's own confidence. ProSA found that when a model is highly confident, it shrugs off rephrasings of the prompt; when confidence is low, small wording changes swing the output wildly Does model confidence predict robustness to prompt changes?. That makes prompt-fragility itself measurable in advance — low confidence is an early-warning signal that this particular prompt is standing on thin ice for this question. Larger models, few-shot examples, and objective tasks all push confidence up and fragility down.
The corpus also reframes the question: failure isn't a property of the prompt alone, it's a property of the prompt-question *fit*. The same reasoning prompt that helps one question hurts another. Instance-adaptive work shows chain-of-thought fails precisely when the question's information doesn't flow into the prompt structure before reasoning begins — for simple questions, step-by-step prompting underperforms a direct question-to-answer path Why do some questions perform better without step-by-step reasoning?. A recommender benchmark found the same: step-by-step reasoning *reduces* accuracy on high-performance models while helping cheap ones, so prompt effectiveness depends on the model tier and task, not on any universal best practice Do prompt techniques work the same across all LLM tiers?. And prompts optimized in isolation from the inference strategy (best-of-N, majority voting) systematically underperform — predictable failure from a mismatch the prompt-writer never accounted for Does prompt optimization without inference strategy fail?.
There's also a hard ceiling worth knowing about, because it bounds the whole question. No prompt can succeed on a question whose answer requires knowledge the model never learned — prompting reorganizes existing knowledge, it can't inject new knowledge Can prompt optimization teach models knowledge they lack?. So one fully reliable prediction of failure is: the question demands facts outside the training distribution. Relatedly, when a user gives too little scaffolding, the model falls back on blended training-data priors and produces generic answers — a predictable collapse rooted in under-specification, not the model's capability Why do large language models produce generic responses to vague queries?.
Finally, some signals are visible only mid-generation rather than before. The fraction of reasoning steps that land in abandoned branches predicts final correctness better than how long the chain is — and those failed branches actively poison what comes after Does failed-step fraction predict reasoning quality better?. If you want to move from predicting failure to preventing it, the corpus offers measurable prompt-quality dimensions to audit a prompt up front Can we measure prompt quality independent of model outputs? and structured critical-question prompting that forces the model to check the warrants chain-of-thought normally skips Can structured argument prompts make LLM reasoning more rigorous?. The through-line: failure is rarely random. It's predictable from answer-improbability, model confidence, prompt-question fit, and knowledge boundaries — and the better you measure those, the earlier you see it coming.
Sources 10 notes
By framing LLMs as autoregressive probability machines, researchers predicted tasks with low-probability target responses would be systematically harder, even when logically simple. Experiments confirmed predictions like backwards alphabet and letter counting.
ProSA found that when models are highly confident, they resist prompt rephrasing; low confidence causes major output swings. Larger models, few-shot examples, and objective tasks all correlate with higher confidence and greater robustness.
Saliency analysis reveals that CoT prompting fails when question information doesn't aggregate into the prompt structure before reasoning begins. For simple questions, direct question-to-answer flow outperforms step-by-step reasoning, showing the optimal prompt depends on question type, not just task category.
A 23-prompt benchmark across 12 LLMs shows rephrasing and background-knowledge prompts boost cheap models, while step-by-step reasoning reduces accuracy in high-performance models. Task structure, not generic best practices, determines which prompts help.
Prompts optimized without knowledge of the inference strategy (best-of-N, majority voting) systematically underperform. Joint optimization of both prompt and inference strategy yields up to 50% improvement across reasoning and generation tasks.
Prompting works entirely within a model's pre-existing training distribution and cannot supply domain knowledge absent from training data. This creates a hard ceiling: no prompt strategy can compensate for missing foundational knowledge, only reorganize what already exists.
Unlike social-media context collapse, which flattens multiple audiences, LLM collapse occurs when users provide insufficient contextual scaffolding and models default to blended training-data priors. This distinction suggests remedies should focus on query verification and user-driven context specification rather than platform controls.
Across 10 reasoning models, the fraction of steps in abandoned branches consistently predicts correctness better than CoT length or review ratio. Failed branches persist in context and bias subsequent reasoning, a phenomenon confirmed through correlation, reranking, and direct causal editing.
Research identifies six evaluable dimensions—Communication, Cognition, Instruction, Logic, Hallucination, and Responsibility—with 20 sub-criteria based on Grice, cognitive load theory, and instructional design. Improvements in one dimension cascade to others, revealing prompt quality as a structured space rather than a flat checklist.
Applying Toulmin's argument model as explicit prompting steps (CQoT) improves LLM reasoning by forcing models to identify warrants and backing rather than skipping implicit premises. The method catches failures that standard chain-of-thought prompting allows.