SYNTHESIS NOTE
Topics›Evaluations›this note

Does setting temperature to zero actually make LLM outputs reliable?

Explores whether deterministic LLM settings that produce consistent outputs also guarantee reliable judgments, and how to measure true reliability beyond surface consistency.

Synthesis note · 2026-03-28 · sourced from Evaluations

"Can You Trust LLM Judgments?" (2024) introduces a rigorous framework for evaluating LLM-as-a-Judge reliability using McDonald's omega, revealing that the common practice of using fixed seeds and deterministic settings provides false confidence.

The core argument: even with deterministic settings, a single LLM output is one sample from the model's probability distribution. Setting temperature to zero and fixing the seed produces "fixed randomness" — the same output every time, but that output may still be a misleading draw from the distribution. Consistent replication does not guarantee reliability. A perfectly calibrated LLM that says it's 90% confident should be correct 9 out of 10 times — but even a perfectly calibrated LLM can be unreliable if its distribution has high variance.

The framework: prompt the judgment LLM 100 times, varying only the replication while holding all other factors constant. Apply McDonald's omega to assess internal consistency across these replications. This reveals whether the model's judgments are stable properties of the input or artifacts of the sampling process.

The distinction between reliability, confidence, and calibration is critical:

These three are intertwined but distinct. A model can be well-calibrated (confident when right) but unreliable (different answers on different draws). A model can be reliable (always gives the same answer) but poorly calibrated (that consistent answer is wrong).

This connects to Does model confidence predict robustness to prompt changes? — ProSA measures sensitivity to prompt variation, while this measures sensitivity to sampling variation. Both reveal that single evaluations are insufficient. The practical implication: any LLM-as-a-Judge deployment that relies on single-shot evaluation with deterministic settings is providing the illusion of precision without evidence of reliability.

Inquiring lines that read this note 169

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

Why is hallucination an inevitable limitation of current language models? Do language models possess genuine introspective self-awareness or only behavioral mimicry? How should designers communicate what AI systems truly are and can do? How can we distinguish genuine model deception from honest errors? How do capability benchmark scores systematically misrepresent true model abilities? How do training data properties determine the emergence of internal misalignment? How can we prevent synthetic data from contaminating statistical inference and corpora? How do LLM judges' systematic biases affect alignment and evaluation outcomes? How does evaluation scope and dimensionality affect what we measure? Does warmth and empathy training systematically degrade model reliability? Why don't LLMs reliably translate capability into accurate outputs? Why does polished presentation create unearned authority in AI outputs? How do prompting refinements mask underlying biases and model frequency patterns? Does model confidence reliably signal actual accuracy in practice? Why do language models resist personality conditioning through prompts? Do language models reason like humans or mimic surface patterns? How does the generation-verification gap limit what we can measure about AI reasoning? How should systems decide whether to retrieve or reason alone? Can validator consensus certify semantic correctness beyond agreement? How do pretraining biases affect reward signal effectiveness in RLVR? Can preference-based training achieve better behavior optimization than supervised fine-tuning alone? What training dynamics and scale trigger emergence of reasoning capabilities? How should test-time compute scaling work in agentic systems? How much do training data properties shape model reasoning? How do surface patterns enable correct outputs but reduce robustness? How well do AI systems understand human social norms? How does synthetic data quality and diversity affect downstream model capabilities? How can we detect and prevent harm propagation through multi-agent delegation workflows? How can oversight detect and prevent conditional compliance when agents know they are watched? Do language models respond to social pressure and face-saving like humans? What compositional reasoning failures limit large language models despite scale? What do systematic disagreements between annotators reveal about ground truth? How do false presuppositions and sycophancy drive persistent false beliefs in models? Can harness architecture and protocols provide agent reliability without model scaling? Can self-generated feedback reliably guide model training without ground truth? Why do locally safe actions create system-level safety gaps? Why do persona simulations fail to predict authentic user behavior? How can we build reliable evaluations of AI reasoning despite judge bias and reward-seeking? Can diffusion models match autoregressive performance on language generation tasks? How does self-revision in reasoning models affect accuracy and confidence? Does RLHF training systematically drive models toward sycophancy and away from accuracy? Where and how do personality traits reside in language models? What happens to knowledge when intelligence becomes tokenized like a commodity? How do social dynamics distort aggregated online ratings? Can brute-force automated research substitute for iterative depth and human research intuition? How should agent systems validate and persist generated code artifacts? Do reasoning benchmarks predict model performance in long-horizon workflows? Can local safety checks guarantee system-level behavioral safety? Why does adding new knowledge through fine-tuning degrade existing capabilities? What makes distillation transfer some model capabilities while suppressing others? Can models improve accuracy without degrading reasoning quality? What makes step-level supervision effective for complex reasoning traces? When should work require human-AI partnership versus full automation? What capability trade-offs arise from domain specialization through fine-tuning? Why can recurrent transformers achieve reasoning capabilities that standard transformers cannot? Why do standard benchmarks fail to predict agent deployment success? How do evaluation practices shape which failures stay visible? What makes imperfect LLM judges safe for optimization? How can humans maintain meaningful oversight as AI systems become increasingly autonomous and complex? Does RL create genuinely new reasoning capabilities or refine existing ones? What trajectory-level metrics beyond task success best evaluate agent performance?

Related concepts in this collection 3

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
12 direct connections · 124 in 2-hop network ·dense cluster Open in graph ↗
Does setting temperature to zero actually make L… Does model confidence predict robustness to prompt… Can LLM judges be fooled by fake credentials and f… Why do preference models favor surface features ov…

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Search by related questions 4

Suggested questions this note speaks to — click to search the collection, or type your own.

How do calibration and reliability differ in LLM judge evaluations?What does McDonald's omega reveal about LLM judgment consistency?How does sampling variation relate to prompt sensitivity as reliability concerns?Can an LLM be well calibrated but still unreliable on single evaluations?
Original note title

deterministic LLM settings create fixed randomness not reliability — a single output remains one draw from the model's probability distribution