admin
This weekend, on September 12th, Dario Amodei, CEO of leading artificial intelligence company Anthropic, stepped into the global spotlight to issue a stark warning and propose a comprehensive, multi-pronged strategy for managing the rapid advancement of frontier artificial intelligence. In a significant address that outlined a three-stage plan, Amodei called for coordinated international efforts to establish limits on the development of the most powerful AI systems, acknowledging the inherent risks and the need for unprecedented collaboration. The initial phase of Anthropic’s proposal centers on an invitation for external evaluators to gain access to the company’s internal operations, mirroring the privileges currently afforded to its dedicated risk assessment teams.
This groundbreaking initiative would grant these independent reviewers access to company hardware, workspace, and direct communication channels with Anthropic employees. Crucially, their contractual agreements would empower them to publish key findings without the company exercising editorial control over their conclusions, subject only to essential legal, security, privacy, and narrowly defined commercial constraints. The objective is to provide an objective test of whether Anthropic’s stated safety commitments translate into tangible decisions during the critical stages of AI training and deployment.
However, Amodei was quick to underscore a significant challenge: internal transparency within one laboratory cannot unilaterally decelerate a field characterized by intense global competition. While Anthropic may open its systems to scrutiny, rival companies and governments are expected to continue their accelerated development trajectories. Recognizing this disparity, Amodei’s second stage advocates for the implementation of regulation and government-mediated coordination among a critical mass of U.S. frontier AI developers. The third and final stage aims to establish verifiable agreements between nations, with a particular emphasis on democracies preserving sufficient strategic autonomy relative to geopolitical rivals like China to effectively pace AI development.
The Complex Interplay of Ownership, Access, and Control
The ongoing debate surrounding AI governance often intertwines three distinct levers of power: nationalization, decentralization, and direct control. Public ownership of AI development fundamentally alters the distribution of economic gains derived from the technology and significantly influences corporate decision-making processes. Independent access, as proposed by Anthropic, determines who has the authority and capability to inspect and audit frontier AI development. Conversely, a legally enforceable halt mechanism directly constrains the velocity at which a covered AI system can advance.
Anthropic’s existing corporate structure already provides its directors with the latitude to consider factors beyond mere shareholder returns. The company operates as a Public Benefit Corporation (PBC), a legal designation that, under Delaware law, mandates its directors to balance the financial interests of stockholders with the well-being of individuals materially affected by the business and its stated public benefit. Furthermore, the establishment of its Long-Term Benefit Trust empowers the board with selection powers designed to safeguard and advance the company’s overarching mission, which prominently features safety and responsible AI development.
While a public-benefit charter can foster safety-minded decisions within Anthropic, it possesses no inherent power to bind competitors who may not embrace the same ethical trade-offs. Amodei’s proposal seeks to address this critical gap by advocating for universally applied common rules and international verification mechanisms, rather than a wholesale transfer of company ownership.
Differentiating Ownership and Control in AI Governance
The nuanced distinction between ownership and control in the context of AI development was recently highlighted by a significant legislative proposal. In June 2026, Senator Bernie Sanders introduced a bill to establish a $7 trillion American AI Sovereign Wealth Fund. This ambitious proposal envisions a 50% public stake in the nation’s largest AI companies, with an independent commission designated to exercise the voting rights associated with this public equity. While currently a proposal rather than enacted law, it illustrates a potential pathway towards partial nationalization of the AI industry.
The injection of public equity could serve a dual purpose: it could redirect a portion of the industry’s substantial economic gains towards public benefit and grant the oversight commission a degree of influence over corporate strategic decisions. However, even with such a fund, crucial elements like capability thresholds, independent external verification, and enforceable stop orders would necessitate separate legal frameworks and regulatory mechanisms.
A recent legal paper, published in August 2026 by Yonathan Arbel, Simon Goldstein, and Peter Salib, delves into the separation of economic claims from decision-making authority in areas where conventional legal rules may prove insufficient. The authors propose a narrowly defined, discretionary, and temporary governmental power to halt frontier AI training or deployment in instances involving catastrophic existential risk or what they term "hard" corporate power – situations where the concentration of power within private entities poses an unmanageable threat. For more conventional concerns such as market monopolies, economic inequality, and other societal harms, they suggest the continued reliance on traditional regulatory and taxation measures.
The concept of a halt order, as proposed by Arbel, Goldstein, and Salib, directly addresses the pacing of AI development more effectively than public equity alone. Under such a framework, the state would not need to own every AI model or operate every research laboratory before it could suspend covered training or deployment activities. For such an authority to claim democratic legitimacy, it would require clearly defined statutory triggers, a high degree of technical expertise within the reviewing bodies, independent and impartial review processes, and strict limitations on discretionary power.
Risks of Centralized Government Control and the Open-Weight Debate
The concentration of power within government entities also presents its own set of risks. The wholesale transfer of every frontier AI laboratory under state ownership could inadvertently place both the development of advanced AI models and the ultimate decision to halt such development within the same institutional framework. Conversely, a bounded halt power, which grants the state emergency intervention capabilities for defined extreme risks while leaving companies largely in private hands, offers a more balanced approach.
Simultaneously, the proliferation of open-weight AI models represents a counter-trend, advocating for broader access to AI systems. Researchers can inspect, adapt, and build upon these models without being solely reliant on a limited number of corporate gatekeepers. In 2024, the U.S. National Telecommunications and Information Administration (NTIA) concluded in a comprehensive report that the available evidence did not warrant blanket restrictions on widely accessible model weights, suggesting a preference for less restrictive measures.
However, the evolving frontier of AI capabilities introduces new complexities to the enforcement of safety measures. The European Commission, for instance, has implemented regulations under the AI Act that require providers of general-purpose AI models deemed to pose systemic risks to conduct thorough risk assessments, mitigate identified dangers, report serious incidents, and maintain robust cybersecurity practices. These obligations extend even to models that are open-source. The Commission has also voiced concerns that mitigation efforts can become significantly more challenging once an advanced AI model has been openly released into the public domain.
The dual nature of open releases – simultaneously expanding outside scrutiny while complicating subsequent enforcement – poses a significant challenge. Replication of AI systems across multiple jurisdictions makes the consistent application of mitigation strategies more difficult. While distributed auditing by numerous institutions can serve as a valuable check against captured regulators or corporations, the unrestricted distribution of frontier AI model weights can undermine the very control points that a lawful pause or halt would necessitate.
The Imperative of Plural Oversight for Public Control Mechanisms
The governance of privately owned AI developers by public rules is being actively explored by jurisdictions like California and the European Union. California’s Senate Bill 53, signed into law in September 2025, mandates that large frontier AI developers publish their safety frameworks, establish a dedicated channel for reporting potential critical safety incidents, and provide protections for whistleblowers. Similarly, the EU’s AI Act imposes risk-management duties on providers of AI models that carry systemic risks, including those that are open-source.
Amodei’s proposal to integrate external evaluators aims to provide a deeper level of access, enabling a more rigorous examination of whether comparable safety duties are effectively influencing internal corporate decision-making. Coupled with a narrowly defined, temporary halt power, this approach would equip public authorities with a critical enforcement option when a covered AI system demonstrably crosses a legally defined risk threshold.
This hybrid structure envisions a collaborative ecosystem where governments establish binding rules for developers of systemically significant AI, independent external evaluators verify compliance with these rules, and public authorities retain the authority to temporarily pause specified training or deployment activities. Crucially, this framework would preserve separate avenues for researchers, whistleblowers, and regulators across multiple jurisdictions to challenge the evidence and ensure accountability.
For such a public brake mechanism to function effectively and democratically, its activation criteria must be intrinsically tied to demonstrated capabilities or significant safety failures. Furthermore, any invoked halt must be subject to review by bodies independent of the office initiating the pause. Explicit rules governing the expiry and renewal of any halt order would also be essential to prevent its indefinite extension. Evaluators must be granted the autonomy to report unfavorable findings, with redactions limited strictly to legitimate legal, security, privacy, and narrowly tailored commercial necessities. These safeguards are paramount to mitigating the risk that a temporary safety intervention could morph into permanent political control over general-purpose AI research.
Under such a common regulatory boundary, private AI development could continue to flourish, provided that frontier systems remain identifiable and enforceable control points are maintained. Independent institutions would then be tasked with inspecting compliance and exposing instances of either corporate or regulatory capture, thereby fostering a more robust and accountable AI ecosystem.
While a public stake in AI companies can effectively redistribute wealth and influence boardroom decisions, ownership alone does not dictate when AI training must cease. Similarly, the open distribution of AI models can broaden access and enhance public scrutiny, but it cannot provide an enforceable stopping rule once advanced model weights have been widely disseminated.
Ultimately, credible pacing of frontier AI development necessitates that every covered developer adheres to the same set of public boundaries. Ensuring democratic legitimacy requires a robust system of independent evaluators, researchers, whistleblowers, and regulators who possess the unimpeded ability to scrutinize the evidence and contest both the established boundaries and any subsequent orders to halt development. This multi-faceted approach, combining corporate responsibility with rigorous public oversight, offers a potential path towards harnessing the transformative power of AI while safeguarding against its most profound risks.
