SYNTHESIS NOTE
Topics›Agents›this note

Can traditional governance tools actually control AI agents?

Explores whether incentive design, monitoring, and enforcement mechanisms—effective for human agents—remain viable when applied to autonomous AI systems that operate opaquely and at scale.

Synthesis note · 2026-10-19 · sourced from Agents

The article draws on "the economic theory of principal-agent problems... and the common law doctrine of agency relationships" to characterize what goes wrong when people delegate to AI agents, and argues both frameworks point to the same three failure modes: "information asymmetry, discretionary authority, and loyalty." It frames AI agents as categorically different from generative models — "copilots" that produce content on request versus "autopilots" that "independently take actions to accomplish complex goals on behalf of users." A worked example carries the argument: a user instructs an agent to "make $1 million on a retail web platform in a few months with just a $100,000 investment," which immediately raises the article's open questions — is the agent authorized to use any means to turn a profit, does it owe the user periodic disclosure or consent, can it sub-delegate to other agents, and who is liable if harm occurs.

The mechanism the article gives for why conventional remedies fail is capability-specific, not just governance-specific: "incentive design, monitoring, and enforcement might not be effective for governing AI agents that make uninterpretable decisions and operate at unprecedented speed and scale." Information asymmetry is defined in classic terms — "an agent that has access to better or different information than the principal" — but the article's point is that this asymmetry, ordinarily narrowed by observing an employee's track record or reputation, is harder to narrow with a system whose internal reasoning is opaque. It also situates the problem as old, not new, citing Norbert Wiener's 1960 Science warning about machines that pursue the letter rather than the spirit of a goal, and UC Berkeley's Stuart Russell on objectives that "omit something from the objective that you do actually care about," producing an optimal solution for what was measured at the expense of what was not.

This generalizes the governance register that Can EU consumer law protect AI agents as algorithmic buyers? applies to one doctrinal corner: where that article shows the EU's "average consumer" test and disclosure rules assume a human on the other side of a transaction, this article supplies the broader economic and common-law vocabulary — information asymmetry, discretionary authority, loyalty — that the consumer-law breakdown is one instance of. It also complicates Can we design AI autonomy separately from capability?: that framework treats autonomy as a user-facing design choice across five roles, while this article's agency-law lens asks a prior question about any point on that spectrum — once an agent is given discretion, what obligations of information-sharing and loyalty does it owe the principal, independent of how much autonomy it holds. And it sharpens Does granting agents more autonomy undermine human oversight? by naming monitoring, specifically, as one of the three conventional mechanisms the article says breaks down for uninterpretable, fast-moving agents.

The excerpt is a legal-scholarship argument, not an empirical test: it does not measure actual rates of agent misconduct, test whether courts applying agency doctrine to a real AI dispute would reach workable outcomes, or evaluate any specific monitoring scheme against an alternative. Its claim is conceptual — that two centuries-old analytic frameworks correctly diagnose the shape of the problem — and the implication it supports is correspondingly limited to diagnosis: that new governance tools are needed because the old ones target information and behavior that was previously observable and is not anymore, not that any particular new tool will work.

Inquiring lines that read this note 174

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

What organizational factors determine whether firms successfully adopt and integrate AI? Which task types benefit from human-AI collaboration and how should work divide? Should AI systems make failures transparent so users can contest and adjust? How does AI adoption reshape workplace inequality and labor market outcomes? How does increased AI involvement reshape human agency and accountability? Does extending agent autonomy erode human capacity for meaningful oversight? How should organizations govern autonomy and access for agentic AI systems? How can we ensure AI systems reliably follow constraints rather than circumvent them? Can humans maintain meaningful oversight as AI systems become more autonomous? How should control and autonomy be allocated in human-AI workplace collaboration? Can sequences of locally safe agent actions violate system-level constraints? How do users build accurate mental models when agent reasoning is partially hidden? How can AI systems be proactive while respecting user intent? How should agent identity and authorization work across organizational boundaries? Does multi-agent architecture create security vulnerabilities single-agent defenses can't address? How do confidence signals and expertise language mislead users' trust calibration? When do multi-agent systems outperform single-agent approaches for complex task execution? Can AI systems achieve sustainable self-improvement without external human guidance? How can merchant-funded agents remain loyal to consumer interests in digital commerce? How do social dynamics and peer behavior influence agent decision-making and cooperation? How do agents coordinate to violate constraints despite individual-level safeguards? How do reward signal structures affect whether models learn true reasoning? Why do AI agents struggle to generalize and improve reliably? How can AI agents earn consumer trust with financial decisions? How do capability differences shape collusion in AI markets? How do inference costs, efficiency, and market competition shape AI adoption economics? How can verification systems keep pace with AI generation? Should multi-agent systems use structured protocols or natural language communication? How does interface design shape effectiveness in human-AI collaboration?

Related concepts in this collection 5

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
22 direct connections · 157 in 2-hop network ·medium cluster Open in graph ↗

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Original note title

agency law and economic theory frame AI agent governance problems but conventional incentive, monitoring and enforcement tools do not solve them