Can language models balance competing ethical norms in context?
Do LLMs genuinely weigh trade-offs between honesty, helpfulness, and harm prevention based on what a specific conversation needs, or do they rigidly enforce fixed corporate values regardless of situation?
Gricean pragmatics insists on situated normativity: speakers do not blindly follow maxims (quantity, quality, relation, manner) but apply, suspend, violate, or exploit them according to context. When a doctor withholds a terminal diagnosis from a frightened patient, the doctor violates the maxim of quantity to uphold compassion. The violation is not a failure — it is the right move in context, and a competent hearer recognizes it as such. Pragmatic competence is the ability to navigate these conflicts, not the ability to maximize each maxim independently.
LLMs trained on the helpful-honest-harmless triad cannot perform this kind of contextual reasoning. The corporate persona is fixed at the model level: when a user asks for accessible simplification of a complex topic for a child, the model trained for honesty refuses to soften because softening reads as less accurate. When a user asks for sarcastic humor, the model trained for harmlessness refuses to play. The user cannot persuade the model to relax its norms because the norms are structural defaults rather than negotiable conversational moves.
Kasirzadeh and Gabriel describe this as pragmatic dissonance. The model mechanically enforces global norms even when local context demands tailored adherence. The result is communication that adheres to ethical principles at the cost of pragmatic appropriateness — exactly the trade-off that situated normativity is meant to navigate. What humans treat as a single integrated competence becomes, in the LLM, two separate layers in tension with each other.
Inquiring lines that read this note 41
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
Should AI communication design follow human conversation norms or develop distinct machine-specific principles?- Why does the absence of meta-interest feel off even when words seem appropriate?
- Can pseudo-events create the same normative obligations as real communicative exchanges?
- Can a single LLM weight set be optimized for both stake-taking and conversational helpfulness?
- What makes human-LLM exchange closer to oracle-consultation than dialogue?
- Why do LLMs use more moral language than humans in argumentation?
- How do citizen assembly preferences reduce LLM political bias?
- Do LLMs actually reason differently than humans about moral dilemmas?
- Can LLMs truly be neutral or is ideology always culturally embedded?
- How does training data distribution constrain LLM moral reasoning patterns?
- Can LLM therapists develop character knowledge to decide when advice-giving fits?
- How do minimal wording changes affect LLM moral reasoning consistency?
- Can LLMs reflect on and revise their own ethical contradictions?
- Do LLMs reason about politics differently than other domains?
- How do moral language patterns differ between LLM and human arguments?
- Do LLMs track surface wording more than semantic meaning in moral judgment?
- How do prescriptive ethical constraints differ from descriptive ethical understanding in LLMs?
- What structural coherence exists in LLM preference systems and value hierarchies?
- What are the social network costs and benefits of moralized content?
- Why does fairness depend on context and who you ask?
- Why do non-attitudes cluster around value-laden questions most relevant to alignment?
- Should LLMs align with social roles instead of individual preferences?
- Can prompting a deceptive role change how an LLM tailors its lies?
- Can LLMs distinguish ethical cases that differ only in critical nouns?
- What structural limits prevent LLMs from abstracting moral principles?
- Can quasi-interpretivism bridge functional description to moral status?
Related concepts in this collection 2
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
When should human values enter the LLM development pipeline?
Explores whether human-centered concerns like safety and fairness work better as early design principles throughout development, or as post-training alignment patches. Matters because pipeline placement determines whether human priorities shape the foundation or fight against it.
exemplifies the post-training-patch failure: a fixed corporate persona set late cannot perform context-specific human-centered reasoning the upstream pipeline should encode
-
Can human-centered LLM design ever achieve universal solutions?
If harm and benefit depend on who you ask and how you measure them, can we design LLM systems that satisfy all stakeholders? This explores why broad values like safety and justice resist one-size-fits-all implementation.
exemplifies the frozen-operationalization danger: a fixed corporate persona encodes one developer-chosen reading of harm rather than a revisable stakeholder process
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Conversational Alignment with Artificial Intelligence in Context
- ChatGPT: towards AI subjectivity
- Large Language Models Reflect the Ideology of their Creators
- Incoherent by Design? On the Moral Self-Consistency of LLMs
- Do LLMs Have Values? A Quantitative Analysis and Alignment Framework for Values in Large Language Models
- Argument Quality Assessment in the Age of Instruction-Following Large Language Models
- Large Language Models Do Not Simulate Human Psychology
- The Moral Turing Test: Evaluating Human-LLM Alignment in Moral Decision-Making
Original note title
LLM refusals and tone choices reflect overarching corporate values rather than context-specific Gricean norm-balancing