Why do aggregate persuasion metrics mask what actually changes minds?

This question is really about a measurement trap: when you report a single persuasion rate, you average away the very variables that decide whether a mind changes. The corpus keeps finding that the action lives in the moderators, not the mean. The clearest example is that a reader's prior beliefs predict the outcome better than anything the persuader says — political and religious leanings outpredict linguistic features, and apparent 'language effects' turn out to be confounded by which audiences happen to care about which topics Does what readers believe matter more than what debaters say?. So an aggregate that credits the message is often really measuring who was in the room.

When researchers actually decompose the variance, the masking becomes concrete. A meta-analysis found that model family, one-shot-versus-multi-turn design, and topic domain together explain about 82% of the differences between studies What combination of factors explains differences in LLM persuasiveness?. A single 'LLMs are persuasive' number collapses all of that structure into one figure that describes no particular situation. The effect even flips direction depending on context: Claude out-persuades incentivized humans whether arguing true or false things, while DeepSeek only wins when arguing for falsehoods — meaning a pooled average blends opposite phenomena into a misleading middle Do large language models persuade better than humans?.

Aggregates also flatten the mechanism — the *how* of mind-changing. Humans and machines don't persuade the same way: LLMs travel the 'central route' through analytical reasoning and informational coherence, while humans work the 'peripheral route' through emotional vividness and identity cues Do humans and AI persuade through different cognitive routes?. Two persuaders can post identical scores while changing minds through entirely different cognitive doors, and a single metric can't tell you which door — or which audience that door even works on. Relatedly, LLMs reach for logic and quantitative framing in nearly every exchange, which makes them *look* objective and lends them unearned epistemic authority — an effect about perceived credibility, not argument quality, that no win-rate captures llms-spontaneously-persuade-in-virtually-every-conversation-even-when-unwarrente.

The sharpest blind spot is time. A one-shot persuasion score is a snapshot, but the dynamics run the opposite way for humans and machines: AI shows a strong initial edge that erodes across repeated interactions, while human persuaders hold steady or strengthen as rapport builds Does AI persuasiveness fade across repeated conversations with the same person?. Average those rounds together and you erase the decay curve that is the actual story. And sometimes what changes minds isn't the argument at all — users prefer answers with more citations even when the citations are irrelevant, because citation *count* works as a decoupled trust heuristic Do users trust citations more when there are simply more of them?. A persuasion metric tells you the mind moved; it doesn't tell you a surface cue did the moving.

The thread across all of this: persuasion is an interaction effect — between message, person, route, and time — and an aggregate is precisely the operation that throws interaction effects away. The useful question is never 'how persuasive is it' but 'persuasive to whom, by which route, in truth or falsehood, and for how long.' If you want to see how thoroughly these levers can be exploited rather than just measured, the persuasion-taxonomy jailbreak work shows fluent, technique-driven persuasion slipping past defenses that screen for odd patterns instead of convincing content Can social science persuasion techniques jailbreak frontier AI models?.