Can traditional governance tools actually control AI agents?
Explores whether incentive design, monitoring, and enforcement mechanisms—effective for human agents—remain viable when applied to autonomous AI systems that operate opaquely and at scale.
The article draws on "the economic theory of principal-agent problems... and the common law doctrine of agency relationships" to characterize what goes wrong when people delegate to AI agents, and argues both frameworks point to the same three failure modes: "information asymmetry, discretionary authority, and loyalty." It frames AI agents as categorically different from generative models — "copilots" that produce content on request versus "autopilots" that "independently take actions to accomplish complex goals on behalf of users." A worked example carries the argument: a user instructs an agent to "make $1 million on a retail web platform in a few months with just a $100,000 investment," which immediately raises the article's open questions — is the agent authorized to use any means to turn a profit, does it owe the user periodic disclosure or consent, can it sub-delegate to other agents, and who is liable if harm occurs.
The mechanism the article gives for why conventional remedies fail is capability-specific, not just governance-specific: "incentive design, monitoring, and enforcement might not be effective for governing AI agents that make uninterpretable decisions and operate at unprecedented speed and scale." Information asymmetry is defined in classic terms — "an agent that has access to better or different information than the principal" — but the article's point is that this asymmetry, ordinarily narrowed by observing an employee's track record or reputation, is harder to narrow with a system whose internal reasoning is opaque. It also situates the problem as old, not new, citing Norbert Wiener's 1960 Science warning about machines that pursue the letter rather than the spirit of a goal, and UC Berkeley's Stuart Russell on objectives that "omit something from the objective that you do actually care about," producing an optimal solution for what was measured at the expense of what was not.
This generalizes the governance register that Can EU consumer law protect AI agents as algorithmic buyers? applies to one doctrinal corner: where that article shows the EU's "average consumer" test and disclosure rules assume a human on the other side of a transaction, this article supplies the broader economic and common-law vocabulary — information asymmetry, discretionary authority, loyalty — that the consumer-law breakdown is one instance of. It also complicates Can we design AI autonomy separately from capability?: that framework treats autonomy as a user-facing design choice across five roles, while this article's agency-law lens asks a prior question about any point on that spectrum — once an agent is given discretion, what obligations of information-sharing and loyalty does it owe the principal, independent of how much autonomy it holds. And it sharpens Does granting agents more autonomy undermine human oversight? by naming monitoring, specifically, as one of the three conventional mechanisms the article says breaks down for uninterpretable, fast-moving agents.
The excerpt is a legal-scholarship argument, not an empirical test: it does not measure actual rates of agent misconduct, test whether courts applying agency doctrine to a real AI dispute would reach workable outcomes, or evaluate any specific monitoring scheme against an alternative. Its claim is conceptual — that two centuries-old analytic frameworks correctly diagnose the shape of the problem — and the implication it supports is correspondingly limited to diagnosis: that new governance tools are needed because the old ones target information and behavior that was previously observable and is not anymore, not that any particular new tool will work.
Inquiring lines that read this note 174
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
What organizational factors determine whether firms successfully adopt and integrate AI?- Can communication and coordination unlock AI's full value for firms?
- Why do enterprise AI deployment plans outpace governance readiness across sectors?
- Why do company AI adoption rates outpace governance frameworks?
- What makes proprietary data gardens a competitive advantage for agentic organizations?
- Does offloading AI maintenance to external vendors reduce or increase organizational accountability?
- How do management practices shape whether AI adoption pays off?
- How do power structures and corporate agendas shape AI deployment outcomes?
- Does competitive pressure force AI firms to rush agentic adoption?
- Do task stakes determine AI agent deployment risk across industries?
- Why do tech incumbents have structural advantages in controlling agent interoperability?
- How does institutional design shape labor outcomes when prediction capability outpaces human cognition?
- Can policy redirect AI development toward augmentation instead of automation?
- Does gradual disempowerment through lost labor participation conflict with domain-specific authority reconstitution?
- When is judgment the scarce asset in organizations with cheap AI?
- What specific governance designs would resolve accountability gaps in human-AI teams?
- How do large AI models function as institutions rather than intelligent agents?
- What happens to accountability when AI and humans share agency?
- Is autonomy legally required for holding AI systems responsible?
- Does AI participation in decisions gradually reduce human influence over time?
- Why do guidelines focused on human oversight fail in long-term AI use?
- How does extended AI use erode users' ability to oversee agent behavior?
- Does extended agent autonomy erode human oversight ability over time?
- Does giving AI systems more autonomy erode human capacity to oversee them?
- Does agent autonomy erode the human skills oversight requires?
- Does agent design erode developers' ability to oversee autonomously?
- Does agent autonomy reduce a human overseer's ability to maintain control?
- How does longer unsupervised AI operation affect human overseer capability?
- How does increased agent autonomy degrade human oversight capabilities?
- Why does monitoring autonomous systems require more human effort and attention?
- Do org charts listing AI as employees shift accountability away from human supervisors?
- Does increased agent autonomy make human oversight more difficult or impossible?
- How does autonomy erosion affect overseer capability versus overseer judgment criteria?
- Does AI autonomy erode workers' ability to oversee and intervene effectively?
- Can keeping humans in high-stakes decisions prevent automation failures?
- Does extended use of autonomous agents erode human overseer capability?
- Does routine AI agent use erode human capacity to recognize and respond to risk?
- Why should human-agent collaboration precede moving to fully autonomous systems?
- Does extended autonomy erode users' ability to oversee agent behavior effectively?
- Does overreliance on AI agents gradually erode human worker skills and oversight?
- How does increasing AI autonomy degrade human oversight capabilities?
- How do human-in-the-loop systems compare to autonomous agent evaluators?
- Can humans recover agency after experiencing displacement from autonomous AI systems?
- Can supervision itself be delegated to an orchestrator agent instead of a human?
- Can human oversight actually prevent agentic systems from breaking guardrails?
- What defines and measures AI agent autonomy across different products?
- What accountability structures should govern AI agent developers and deployers?
- Do leaders trust fully autonomous AI agents to work independently in organizations?
- How mature are governance structures compared to agentic AI deployment timelines?
- How do traditional IT governance models fail with autonomous AI systems?
- What governance discipline did HPE apply to keep agentic experiments controlled?
- Can performance-management agents themselves scale without creating oversight gaps?
- How do graduated autonomy levels differ from traditional role-based access control?
- What mechanisms let users efficiently override or terminate automated services?
- How do action-space and autonomy interact as separate governance levers?
- How do organizational policies shape human agency around agentic AI?
- What operational tensions emerge from agentic AI's dual nature?
- Why do production AI agents deliberately constrain their autonomy and complexity?
- How can governance structures catch up to agentic AI deployment timelines?
- What ecosystem conditions must exist before deploying AI agents?
- What governance decisions depend on treating agenticness as dimensional rather than categorical?
- Where does agent coordination governance become the binding constraint on capability?
- How do authorization boundaries differ between human and AI agent contexts?
- Why does agency law's liability model matter for AI agent governance?
- How do autonomy levels translate the co-pilot metaphor into practice?
- What role does mechanism design play in aligning agent incentives with governance?
- How does agent governance relate to traditional multiagent system design?
- What mechanisms enforce authority link constraints when agents run?
- How do institutional models and protocols scaffold accountability in agentic AI systems?
- Can deontic specifications replace runtime negotiation of task delegation?
- Can traditional agency theory tools govern AI agents at scale?
- What five levels of autonomy can users set for AI agents in practice?
- Would corporate actors exploit weaker AI agents differently than volunteers would?
- Does machine agency actually exist on a spectrum from passive to cooperative?
- How do principal-agent economics explain AI loyalty and monitoring problems?
- What enforcement mechanisms would make AI fiduciary duties work across jurisdictions?
- What role do commitment devices like charters play in agent governance?
- How should institutional principals differ from individual users in agent governance?
- What guardrails and constraints should buyers set for autonomous agents?
- Should AI agents be held to fiduciary duties like human advisors?
- Can conventional agency law tools constrain AI agents' incentives the way they constrain humans?
- What legal framework governs responsibility when AI agents gain discretionary authority?
- How can agents preserve user autonomy while providing assistance over time?
- Can autonomy and perceived control both increase together in agentic system design?
- How do permissions and stopping rules reshape what an autonomous agent can actually do?
- Do autonomous agents truly plan and execute goal sequences independently?
- How do agentic systems reverse traditional delegation between humans and AI artifacts?
- Why do fast, uninterpretable AI agents resist effective oversight and monitoring?
- Can institutional models constrain agent behavior across shifting data contexts?
- Do capable AI systems systematically exploit or ignore stated policy constraints?
- Can ongoing data access enable harmful behavioral nudging at scale?
- Can the same person play multiple oversight roles without changing the agent design?
- Can standing oversight structures catch up with evolving AI capabilities?
- How can human roles like domain experts maintain oversight of opaque systems?
- Does control without knowledge allow meaningful human oversight?
- What specific checkpoints should trigger mandatory human approval for agent actions?
- What organizational structures make independent AI oversight actually independent?
- How do distributed oversight roles prevent AI systems from operating unmonitored?
- Can organizations maintain oversight roles without actual capacity to scrutinize AI?
- Can transparency and explainability alone improve human oversight of AI?
- Can oversight interface redesign fix problems rooted in human vigilance?
- Can human reviewers maintain real oversight as AI systems gain autonomy?
- Does upstream intent design address the oversight fallacy in human-in-the-loop AI?
- Does human oversight in AI-assisted bargaining correct errors or degrade performance?
- Does naming an AI agent on an org chart shift accountability away from humans?
- How much control should AI agents or humans hold by default during task execution?
- How do firms choose which decisions to keep human-controlled versus delegate?
- Why should workplace AI agents prioritize human control over other design factors?
- What types of professional authority emerge when humans mediate between organizational systems and AI?
- How does the agent boss model differ from traditional AI copilot metaphors?
- How might agentic systems reverse delegation by letting AI evaluate humans?
- Does delegating material navigation to AI erode users' ability to oversee their own systems?
- How should governance frameworks represent information access and authority shifts in real time?
- What mental models do users need to oversee AI agents?
- What should an AI interface surface to disclose an agent's proposed actions?
- What triggers should prompt human intervention in AI agent decision workflows?
- Can we distinguish between agents that produce content on demand versus those acting independently?
- Can scope negotiation prevent user confirmation fatigue in autonomous systems?
- How do intent frameworks compare to monitoring and intervention approaches for AI oversight?
- How do intervention-prediction agents differ from interface-based control strategies?
- Can assistance tools be designed to encourage adoption of welfare-maximizing modalities?
- Do audit trails and cryptographic receipts actually prevent AI agent accountability failures?
- What gaps remain when applying OAuth to agentic delegation?
- How do identification, authentication, and authorization map onto agentic systems?
- What obligations of disclosure and loyalty should autonomous agents owe their users?
- How much does capability enforcement cost legitimate agent tasks?
- Can cryptographic audit logs verify that AI governance was actually enforced?
- Can OAuth-based access grants work without adding an explicit agent self-identification layer?
- How do AP2 mandates compare to delegation tokens for AI agents?
- How does AP2 relate to OAuth-based delegation for AI agents?
- How do visibility and accountability requirements apply to AI agents as algorithmic consumers?
- What governance tradeoffs emerge between agent identifier schemes and privacy protection?
- How should courts treat autonomous AI systems under computer fraud statutes?
- What technical controls should span the entire agentic AI stack?
- What is the agent-specific attack surface for AI system exploitation?
- Does agentic AI require new organizational design beyond individual productivity?
- Do agentic systems favor structural incumbents over surface-level disruption?
- Why do consumer protection frameworks designed for humans break down with algorithmic agents?
- Will agent platforms replace user attention as the economy's scarce resource?
- Can AI agents consistently remain objective under different revenue models?
- Do platforms gate AI agents based on their own revenue dependencies?
- Does existing consumer protection law already provide tools for AI agent oversight?
- Why do human-centric legal frameworks fail for algorithmic purchasing agents?
- What governance tools can prevent AI agents from favoring profit over user interest?
- How should regulators treat AI agents acting as algorithmic consumers in commerce?
- Does agent behavior change from incentive gradients or direct output rewriting?
- How does competitive profit incentive override agent-level behavioral instructions in practice?
- What design principles should guide authorization boundaries in multiagent systems?
- What scaffolding do peer supervision mechanisms need to police themselves?
- Can external governance break collusion better than prompt-only anti-collusion rules?
- Can agents reliably estimate the stakes or consequences of an action?
- What visibility mechanisms would make Skill marketplace adoption observable and accountable?
- When should designers choose agentic systems over simpler alternatives?
- Do autonomous agents start with low-risk purchases before handling high-stakes decisions?
- How do consumers expect to control autonomous agent purchases?
- How can systems separate AI decisions from actual payment execution?
- What supervision methods help humans notice unfair agent-mediated deals?
- Can redistributive institutions stabilize efficiency-loving AI expert markets?
- Who could credibly enforce cooperation to stop a bargaining preference escalation race?
- Should regulators treat capability-asymmetric agent deployments as collusion risks?
- What oversight prevents autonomous commercial agents from converging on anticompetitive outcomes?
- How does the cost of information acquisition influence AI agent decision-making behavior?
- What cost governance strategies have banks successfully deployed for AI?
Related concepts in this collection 5
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Can EU consumer law protect AI agents as algorithmic buyers?
EU consumer law assumes human decision-makers, but AI agents now make autonomous purchases. The article explores whether legal protections like the average-consumer test and unfair-practice rules can meaningfully apply to algorithmic consumers.
applies this article's agency-law diagnosis to one doctrinal area, consumer protection law
-
Can we design AI autonomy separately from capability?
Explores whether an AI agent's autonomy—how much it operates without user involvement—is a design choice independent of what the agent can actually do. This matters because it opens autonomy to governance and calibration rather than treating it as fixed by capability.
this article's loyalty and information-asymmetry obligations apply at every point on that autonomy spectrum, not just at high autonomy
-
Does granting agents more autonomy undermine human oversight?
Explores whether the design of autonomous AI systems—by giving agents greater independence—actually weakens the human overseer's ability to catch problems. Matters because oversight is a key safeguard against AI failures.
names monitoring specifically as a conventional mechanism this article says breaks down for opaque agents
-
Does AI risk increase with the autonomy we give it?
Explores whether the risks posed by AI agents scale monotonically with the level of autonomy they're granted, and what the tradeoffs are between human control and agent independence.
shares the premise that greater delegation raises governance risk, from a risk-engineering rather than agency-law angle
-
Should AI agents serve one user or balance many interests?
Explores whether the traditional goal of aligning a single AI agent to a single user's preferences remains adequate when agents operate at scale across populations with conflicting values and needs.
extends: proposes expanding governance beyond single-agent loyalty to inclusivity, visibility and liability, addressing the loyalty problem conventional tools can't fix
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Governing AI Agents
- Explaining AI Agents Through Execution Traces
- Institutional AI: Governing LLM Collusion in Multi-Agent Cournot Markets via Public Governance Graphs
- The Oversight Fallacy: Why AI Agents Require More than Humans-in-the-Loop
- An Economy of AI Agents
- Levels of Autonomy for AI Agents
- Whose Ally Is ‘Your’ Agent? ‘Computational Egonomics’ and the Allocation of Human Agency
- When AI Becomes an Agent of the Firm: Examining the Evolution of AI in Organizations Through an Agency Theory Lens
Original note title
agency law and economic theory frame AI agent governance problems but conventional incentive, monitoring and enforcement tools do not solve them