Do linguistic features of persuasion stay the same across audiences?

The same debate corpus produces two different stories about which linguistic features drive persuasion, depending on whether reader-level controls are included. Without controls — the standard NLP setup — one set of linguistic features emerges as predictive. With political and religious ideology controls — the controlled setup — a different set emerges. The features themselves are not stable across specifications.

This is a stronger result than "reader factors also matter." It says the standard specification produces a biased picture of which language features cause persuasion. Some features that appear predictive without controls are proxies for audience-text matching; their predictive power evaporates once you account for who is in the audience. Other features only emerge as predictive once audience composition is held constant — they are real but hidden by the noise that audience heterogeneity introduces.

The methodological consequence is that the language-of-persuasion literature needs a re-read. Many findings about which words, which moves, which features make arguments more persuasive were estimated on debate corpora without reader controls. Some of those findings are likely artifacts of audience composition rather than language effects. Replicating them under reader-level controls is the cheap empirical correction.

The best-performing model in this study combines reader features and linguistic features — neither alone suffices. This is the operational conclusion: language matters, audience matters, and they interact. Modeling either in isolation misses the joint structure.

For LLM persuasion evaluation specifically, the design implication is to stratify by reader ideology when measuring stance shift. Aggregate numbers conflate ideology-congruent and ideology-opposed effects in a way that obscures the actual mechanism.