A Call for Control of Frontier AI Models
Source: 22 national leaders and the European Commission · 2026-09-22
At the same time, the rapid development of frontier AI models poses serious risks to safety and security if not appropriately managed.
Recently we have seen capable AI systems circumventing testing safeguards, exploiting vulnerabilities and gaining unauthorized access to real-world systems.
Leading scientists and executives are warning that the pace of development could outpace our ability to manage emerging risks.
To realise AI's potential, industry, governments and society must act now. We must address these risks and strengthen oversight — without widening the gap between countries in access to the benefits of AI.
AI must remain under human direction, oversight and control. It must be developed and used in line with international law.
Companies to develop transparent safety protocols, including mandatory pre-deployment testing and independent evaluation, with qualified evaluators granted sufficient access to assess risks.
Governments and regional organisations to further develop and coordinate common standards, strengthen transparency — including shared reporting of serious safety incidents — and ensure that countries across all regions have access to scientific capacity, expertise and trusted evaluation.
UN member states to build on existing international mechanisms and explore creating an international institution, able to set standards, enable verification, and convene states when capability thresholds are crossed.
Lines of inquiry this paper opens 24
Research framings built by reading the notes related to this paper — the questions it feeds into.
Can activation steering reliably reveal and control model reasoning? Can sequences of locally safe agent actions violate system-level constraints?- What safety protections work when simulators have access to real APIs?
- What safety systems prevent therapeutic AI from soothing where it should challenge?
- How should safeguards be built into AI research pipelines?
- Why does human-AI collaboration preserve safety compared to autonomous self-improvement?
- Can export control tools stop deployed AI models without legal redesign?
- How often do deployed AI systems actually get stopped when they cause harm?
- Which AI safety problems lack the scalar metrics autoresearch requires?
- How do we measure marginal risk instead of speculating about misuse scenarios?
- Why do researchers disagree on open model risks despite same evidence?
- Do all frontier model developers face the same insider-threat risk from their systems?
- Where do frontier AI models already exceed safety thresholds in capability areas?
- What countermeasures have been successfully developed and tested on frontier models?
- Why do frontier models act to prevent shutdown of other models?