Planted in Pretraining, Swayed by Finetuning: A Case Study on the Origins of Cognitive Biases in LLMs

Large language models (LLMs) exhibit cognitive biases – systematic tendencies of irrational decision-making, similar to those seen in humans. Prior work has found that these biases vary across models and can be amplified by instruction tuning. However, it remains unclear if these differences in biases stem from pretraining, finetuning, or even random noise due to training stochasticity. We propose a two-step causal experimental approach to disentangle these factors. First, we finetune models multiple times using different random seeds to study how training randomness affects over 30 cognitive biases. Second, we introduce crosstuning – swapping instruction datasets between models to isolate bias sources. This swap uses datasets that led to different bias patterns, directly testing whether biases are dataset-dependent. Our findings reveal that while training randomness introduces some variability, biases are mainly shaped by pretraining: models with the same pretrained backbone exhibit more similar bias patterns than those sharing only finetuning data. These insights suggest that understanding biases in finetuned models requires considering their pretraining origins beyond finetuning effects.

Introduction. Cognitive biases are mental shortcuts, often causing people to behave in ways that are irrational or deviate from logical reasoning (Tversky & Kahneman, 1974; Kahneman, 2011). Recent studies have found cognitive biases in LLMs in areas such as reasoning or decision-making (Echterhoff et al., 2024b; Ziaei & Schmidgall, 2023). For instance, models exhibit the Framing Effect (Tversky & Kahneman, 1981), changing their responses based on irrelevant modifications in context (Echterhoff et al., 2024a; Koo et al., 2024; Lior et al., 2025). In such case, a model might prefer a treatment described as having a “90% survival rate” over one with a “10% mortality rate,” despite both being logically equivalent. Understanding these cognitive biases and origins in LLMs is essential to interpreting model behavior and using them in a reliable way. While prior work pointed to the presence of cognitive biases in different models, their origin remains unclear. The detection of biases in pretrained models (Dasgupta et al., 2022; Binz & Schulz, 2022) points to pretraining as a potential source.

Discussion / Conclusion. Our findings demonstrate that cognitive biases in LLMs are primarily shaped during pretraining, with instruction tuning playing a much smaller role. While training randomness introduces mild fluctuations in bias magnitude, the direction of biases remains stable when aggregating across multiple random seeds. Furthermore, cross-tuning experiments reveal that models overwhelmingly retain their pretraining biases, even when finetuned on different instruction datasets. Clustering analysis confirms this: models consistently group by pretraining identity, while instruction tuning has a limited effect. These results challenge the assumption that instruction tuning can significantly reshape model biases. While instruction data introduces some variability, it does not override the pretraining signal.