Does cheap AI simulation break the credibility of costly signals?

The paper argues that generative AI makes it cheap to simulate the output of human mental effort, and that this undermines "mental proof": observable actions used to certify unobservable mental facts such as intentions, values and states of knowledge. The introduction puts the shift plainly: "anyone can now cheaply and convincingly simulate the output of human mental effort across an unprecedented variety of tasks." Mental proofs let people make credible claims in low-trust settings, where "cheap talk" fails and reputation, norms and formal punishment are unavailable. The excerpt names two mechanisms that sustain them, signaling theory from economics and biology and proof-of-knowledge protocols from computer science, and says both "rely on implicit assumptions about the cost structure of organic mental activity."

The mechanism is cost. In the signaling account, a behavior is credible because it is expensive to fake: "The apparent downside of signaling behaviors are precisely what establish their credibility." Generative AI lowers the cost of producing the observable output. Once that output is nearly free, a deceptive type can send the signal too, and the cost structure that kept signals honest no longer does its work. The excerpt develops the signaling case in detail, through Spence's degree and Zahavi's peacock tails, but only names the proof-of-knowledge mechanism, so it does not show how that second mechanism breaks. The discussion adds that mental proof matters most where honesty cannot be enforced, so the resulting harm "will disproportionately impact those who are not already embedded in high-trust networks and formal institutions."

The nearest notes approach the same territory from measurement. The Why do people share more openly with machines than humans? note reports that face-saving and impression-management goals drop out of human-machine talk. This excerpt makes a different kind of claim, a theory of why costly observable behavior carries social information at all, and it does not mention those goals. The Do AI peers influence human dishonesty like human peers do? note measures how AI peers shift dishonest reporting. The paper's argument runs through a different channel: AI need not push anyone toward dishonesty for mental proof to erode, only make the costly evidence cheap to produce. The sibling note takes up whether that erosion is always a loss.

The excerpt establishes a mechanism, not a magnitude. It reports no experiment, survey or sample, and the worked examples it announces, sincere apology and subculture formation, are not in the excerpt. Its claim that AI has begun to disrupt college assessment, online dating and sincere apologies rests on citations it names (Fitria, 2023; Cardon et al., 2023; Wu and Kelly, 2020; Glikson and Asscher, 2023) but does not summarize. The therapy case is raised as an open problem: "we do not yet understand how this might undermine the therapeutic alliance." The implication is to read the framework as a lens for where harm should concentrate, in settings where honesty cannot be enforced, and as a hypothesis to test rather than a measured effect.