Do Reasoning Representations Help Humans Evaluate LLM Outputs?

Reasoning representations are increasingly used as explanations for large language model outputs. Yet they are typically evaluated with model-centric criteria, such as answer accuracy and faithfulness, leaving it unclear whether they help people evaluate model responses. In this work, we study reasoning representations as human-facing interfaces rather than proxies for model reasoning ability. We conduct a controlled human study of six reasoning formats across tasks of varying complexity, supported by a web-based framework that randomizes task domains, problem instances, and representation order. The study collects fine-grained judgments of structural understanding, error detection and localization, and trust calibration. Our study shows a mismatch between perceived preference and support for human evaluation. Participants prefer planning- and decomposition-based representations, but simpler chain-of-thought traces better support verification, trust, and interpretability. Preferred representations also introduce calibration risks, with more false alarms on correct traces and high trust despite low willingness to verify.

Introduction. Large language models are increasingly used for tasks that require not only final answers but also reasoning traces that help users evaluate the outcomes. Methods such as chain-of-thought prompting (Kojima et al., 2022; Wei et al., 2022), selfconsistency (Wang et al., 2023b), planning (Wang et al., 2023a), decomposition (Zhou et al., 2023), and template-based reasoning (Yang et al., 2024) expose such traces in different forms. While these reasoning representations were originally introduced to improve model performance, they are now used as explanations for assessing whether a model response is correct and trustworthy. This shift creates an evaluation mismatch. Reasoning representations are treated as user-facing explanations, but evaluated primarily with modelcentric criteria.

Discussion / Conclusion. This work studies reasoning representations as interfaces for human oversight, rather than as modelcentric indicators of its ability alone. Through a controlled human evaluation of six reasoning representations, we show that user preference, verification performance, and trust calibration do not always align: users favor more structured planningand decomposition-based ones, while simpler CoT traces better support error detection and localization. Preferred formats can also produce miscalibrated judgments, suggesting that perceived usefulness does not guarantee effective verification. Our findings challenge the assumption that more visible or structured reasoning uniformly improves human evaluation. As reasoning becomes embedded in user-facing LLM systems, its representations should be designed and evaluated for the human judgments they support—understanding, verification, and trust calibration—rather than for modelcentric criteria such as accuracy or faithfulness.