Do language models learn abstract grammar or cultural speech patterns?
LLMs might learn more than grammar rules—they could be learning who says what to whom and when. This matters because it changes how we understand what biases and persona effects actually represent.
The computational structuralism framework (2026) makes a subtle but important distinction: LLMs trained on web text do not learn "language" in Saussure's abstract sense but learn culturally and socially situated linguistic action. They learn to discriminate not just grammatically correct from incorrect statements but socially and contextually correct from incorrect ones — which voices are likely to make which statements in response to which situations, and conditional on those statements, how audiences are likely to respond.
This reframes several debates:
For alignment: LLMs do not have "opinions" but have learned the statistical landscape of who holds which positions and when. Their outputs are not assertions but reproductions of situated discourse patterns. This connects to Does AI generate diverse claims or diverse perspectives? — the claims are there because the training data associates them with specific cultural positions, not because the model advocates for them.
For bias: LLM biases are not random but reflect the structured correspondences in training data between social positions and linguistic registers. The "modest banker" and "rebellious artist" reproduce the classification schemas that Bourdieu described — transposable dispositions applied across domains. This is not a training failure but a feature of learning culturally situated language.
For persona simulation: Research showing LLMs can simulate political perspectives or social personas (Argyle et al. 2023; Park et al. 2024) is expected rather than surprising under this framework — these persona structures are exactly what the model's compression captures.
For mechanistic interpretability: The structuralist lens suggests that mechanistic interpretability could answer long-standing questions in cultural sociology: how are ideologies composed from simpler features? Can opposing schools of thought be decomposed into their constitutive features? This connects LLM research to the sociological program of relational analysis (White, Breiger, Bourdieu).
The key contrast with Can language models learn meaning without engaging the world?: that note covers the theoretical architecture; this note covers the empirical consequence — what LLMs actually learn is not "a language" but "how a culture speaks."
Inquiring lines that use this note as a source 4
This note is a source for these synthesized inquiries. Follow a line forward into its question, or open it to trace back to all of its sources.
- What is the difference between learning discourse patterns and learning abstract language?
- Can LLMs truly be neutral or is ideology always culturally embedded?
- Do LLMs learn linguistic generalizations or just surface-level frequency patterns?
- How much cultural knowledge exists only in unwritten social rules?
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Computational structuralism: Toward a formal theory of meaning in the age of digital intelligence
- Large Language Models Reflect the Ideology of their Creators
- In-context learning agents are asymmetric belief updaters
- SDPO: Segment-Level Direct Preference Optimization for Social Agents
- Conversational Alignment with Artificial Intelligence in Context
- The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversation
- Word Meanings in Transformer Language Models
- Lil-Bevo: Explorations of Strategies for Training Language Models in More Humanlike Ways
Original note title
LLMs learn culturally situated discourse patterns not abstract language — they encode which voices make which statements in which situations