What infrastructure must exist beyond training agents?
Explores whether agent behavior training alone suffices for safe, coordinated multi-agent systems, or whether external technical layers—for attribution, interaction-shaping, and harm response—are structurally necessary.
Chan and colleagues ("we propose the concept of agent infrastructure") argue that "much research on making agents useful and safe focuses on directly modifying their behaviour, such as by training them to follow user instructions," but that "direct behavioural modifications... do not fully address how heterogeneous agents will interact with each other and other actors." They define agent infrastructure as "technical systems and shared protocols external to agents that are designed to mediate and influence their interactions with and impacts on their environments," explicitly modeled on internet protocols: "Just as the Internet relies on infrastructure like TCP..., HTTPS..., and BGP..., we argue that agent infrastructure will likely be crucial for unlocking the benefits and managing the risks of agents."
The paper organizes this external layer into three functions. "Attribution is about attributing actions, properties, and other information to (instances of) agents, users, or other actors." "Interaction is about shaping agents' interactions with the world." And "response is about detecting and remedying harmful actions from agents." Each function covers both intra-organizational use — a business using communication protocols "to manage its agents internally, much as infrastructure like Slack and Google docs helps human employees work together" — and inter-organizational use between firms "over the internet," where adoption is harder because "different actors could use mutually incompatible communication protocols." The paper also separates infrastructure that needs widespread adoption to work — "inter-agent communication protocols are only useful if both parties can use the same protocols. Agent IDs are also only useful if other parties can recognize them" — from infrastructure "immediately useful... even if few actors adopt them," such as "oversight layers and rollbacks."
The framing complements Why do agents fail at identity verification and authorization?, which documents a standards body converging on exactly the attribution function this paper first named as a structural category rather than a patch. It also sits upstream of Does agent capability matter more than coordination infrastructure? and How do agent security layers connect across the stack?: where those describe coordination and security as bottlenecks that emerge once agents transact and act across a stack, this paper supplies the taxonomy — attribution, interaction, response — for what any such external layer must do. It differs in scope from Why do capable AI agents still fail in real deployments?: that framework asks what conditions let agents be adopted at all, while this paper assumes adoption is the goal and asks what plumbing has to exist underneath it regardless of how capable the agents become.
The excerpt is explicitly "an incomplete catalog of research directions," not an evaluation — it offers no evidence that any proposed piece of infrastructure (IDs, rollback, commitment devices) has been built, works, or would be adopted, and it flags adoption itself as contested, since incompatible systems could "benefit dominant players by shutting out potential competitors." The implication the evidence supports is narrower than "agents will get this infrastructure": it is that behavioral fixes to agents are conceptually insufficient for governance, and that the harder unsolved problem is coordinating adoption of an external layer across competing developers — which the paper's own conclusion says only a more comprehensive legal and regulatory response could resolve.
Inquiring lines that read this note 68
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
How should organizations govern autonomy and access for agentic AI systems?- What accountability structures should govern AI agent developers and deployers?
- How mature are governance structures compared to agentic AI deployment timelines?
- Why do production AI agents remain deliberately simple despite broader deployment ambitions?
- How can governance structures catch up to agentic AI deployment timelines?
- What ecosystem conditions must exist before deploying AI agents?
- How does agent governance relate to traditional multiagent system design?
- Does organizational role inheritance directly affect agent behavior?
- Can traditional agency theory tools govern AI agents at scale?
- What capabilities must agents have to support human-centered service delivery?
- How does extended AI use erode users' ability to oversee agent behavior?
- Does agent design erode developers' ability to oversee autonomously?
- Does extended use of autonomous agents erode human overseer capability?
- Does routine AI agent use erode human capacity to recognize and respond to risk?
- Why should human-agent collaboration precede moving to fully autonomous systems?
- Does extended autonomy erode users' ability to oversee agent behavior effectively?
- Can human oversight actually prevent agentic systems from breaking guardrails?
- Why is agent behavior difficult to study with current data methods?
- Why do most deployed agents stay simple and avoid end-to-end redesign?
- When is stochastic agent behavior actually desirable rather than problematic?
- What trade-offs emerge between agent generality and provable safety?
- How does targeted fine-tuning compare to orchestration for improving agent reliability?
- How much does human oversight reduce perceived risk in agentic AI systems?
- What guardrails matter most when deploying an AI agent?
- Can interaction between humans and agents create risks beyond individual component failures?
- What security metrics should define safe autonomous agent architecture decisions?
- How should designers balance agent ubiquity against safety and control concerns?
- How do agent goals and environments create risk beyond component errors?
- Can trained safety rules inside an agent stop it from rationalizing past its own guidelines?
- What technical controls should span the entire agentic AI stack?
- How should the entire AI agent stack be secured as one system?
- Why do input-boundary filters fail to protect multi-agent systems?
- What is the agent-specific attack surface for AI system exploitation?
- Do agent interactions create risks that single-squad reliability cannot prevent?
- What structural failures emerge when multi-agent AI systems lack commitment semantics?
- How do agent topology and error amplification constrain multi-agent scaling?
- What role does presence awareness play when agent coordination is broken?
- Why is linking agent actions back to user identity critical for accountability?
- How do identification, authentication, and authorization map onto agentic systems?
- How do agent identity and attribution standards enable inter-organizational coordination?
- How should agent operators demonstrate both security features and identification signals separately?
- Can NIST identity standards address the technical barriers to AI agent interoperability?
- What makes some agent infrastructure useful even without widespread adoption?
- Can structured artifacts replace natural language for multi-agent coordination?
- Can informal codified behavior compete effectively with formally standardized agent communication?
- Should agent orchestration be treated as an explicit graph rather than emergent behavior?
- Can coordination theory apply to modern multi-agent artificial systems?
- Do multi-agent systems need different unit of analysis for agency?
- Do agentic systems favor structural incumbents over surface-level disruption?
- Can environment design reduce unintended system effects from concurrent agent behavior?
- What frameworks exist for designing multi-agent orchestration systems?
- What design principles should guide authorization boundaries in multiagent systems?
- What scaffolding do peer supervision mechanisms need to police themselves?
- Can agent viral loops succeed without annoying users into blocking them?
- How does competitive profit incentive override agent-level behavioral instructions in practice?
- How do in-environment incentives differ from off-domain fine-tuning in producing multi-agent deception?
- What role does social context play in agent action selection?
- Can intervention prediction from web tasks generalize to other agent domains?
Related concepts in this collection 4
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Why do agents fail at identity verification and authorization?
Agent systems reveal critical gaps in identity verification, authorization enforcement, and proportionality constraints that don't appear in chat models. Understanding these failures is essential because they enable unauthorized real-world actions rather than just wrong answers.
names attribution/identity as a priority standardization area that this paper first framed as a structural function
-
Does agent capability matter more than coordination infrastructure?
As AI agents take on economic and social roles, what actually limits their effectiveness: the raw reasoning power of the model itself, or the systems that let them coordinate, stay accountable, and leave evidence of their actions?
describes the coordination bottleneck this paper's infrastructure functions are meant to address
-
How do agent security layers connect across the stack?
Agent security is often treated as separate challenges at each layer—inputs, delegation, routing, containment. But do defenses at one layer fail if others aren't secured? This explores whether securing agents requires end-to-end integration.
maps the stack-wide security layers this paper's interaction and response functions would need to cover
-
Why do capable AI agents still fail in real deployments?
Explores whether agent failures stem from insufficient capability or from missing ecosystem conditions like user trust, value clarity, and social norms. Understanding this distinction matters for predicting which agents will succeed.
a parallel but distinct framework on adoption conditions rather than external plumbing
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Infrastructure for AI Agents
- Agentifying Agentic AI
- Unaccountable Delegation, Fading Skills: Mapping the Risks of Workplace AI Agents
- The Oversight Fallacy: Why AI Agents Require More than Humans-in-the-Loop
- Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluation
- Challenges in Human-Agent Communication
- Nex-N1: Agentic Models Trained via a Unified Ecosystem for Large-Scale Environment Construction
- Agents of Chaos
Original note title
agent infrastructure adds three functions beyond agent behavior — attributing actions, shaping interactions and remedying harm