The prototype that nobody codified

On November 1, 2023, at the UK government’s first AI Safety Summit at Bletchley Park, Dario Amodei stood up and said two sentences that the rest of his governance position rests on. “RSPs are not intended as a substitute for regulation, but rather a prototype for it.” And: “We’d like to encourage a race to the top in RSP-style frameworks.”

The first sentence set a deadline. A prototype is something a regulator codifies later. If government rules never arrive, what Amodei pitched at Bletchley was not a prototype but a permanent state he had committed his company to in advance.

Nearly three years on, no US federal rule has codified it. Binding law has arrived elsewhere, in other forms. The EU AI Act’s obligations for providers of general-purpose models have applied since August 2025, and since August 2026 the European Commission’s AI Office can require access to a model, run its own evaluations and order a provider to restrict availability. California’s SB 53, signed in September 2025, requires large frontier developers to publish their own safety frameworks and report critical safety incidents. In Washington, the closest thing to government engagement is a voluntary testing channel — labs can submit models to NIST’s Center for Advancing Innovation and Standards for Super Intelligence (CAISSI; as the Center for AI Standards and Innovation, or CAISI, it signed the May 2026 testing agreements) for pre-release evaluation, but the examiner cannot delay a launch.

The prototype, however, has propagated: OpenAI’s Preparedness Framework copied the shape in December 2023, and DeepMind’s Frontier Safety Framework followed in May 2024. All three frameworks have since been amended at least once. Critics on LessWrong and the EA Forum read each amendment as a softening, a loosening of the binding “if/then” structure that made the original commitment legible. Anthropic’s own RSP version 3.0, published in February 2026, drew the same charge from the same critics that spring: that capability-triggered rules had been replaced, in places, with internal discretion. Anthropic has revised the policy since; version 3.4, effective July 8, 2026, changed the automated-R&D threshold and the rules for risk reports and external review.

What an RSP actually is

A Responsible Scaling Policy is a document a frontier lab writes about itself. Anthropic’s first version, published September 2023, organized the policy around a ladder of capability thresholds called AI Safety Levels, or ASL — borrowed by name from the four-level biosafety classification used in laboratory medicine for handling pathogens. ASL-1 covered systems posing no meaningful catastrophic risk, such as a 2018 language model or an AI system that only plays chess. ASL-2 covered the frontier models of 2023, capable but not, in the policy’s own assessment, “operationally useful for catastrophic misuse.” ASL-3 was the next tripwire — a model crossing the catastrophic-misuse threshold, at which point the lab committed in advance to security strong enough to make weight theft by non-state actors prohibitively expensive, and to deployment safeguards aimed at known abuse pathways. ASL-4 sat above, where the model approaches autonomous human-level capability and most of the mitigation work was, by Anthropic’s own admission, still to be invented.

Anthropic activated its ASL-3 protections in May 2025, with the launch of Claude Opus 4, as what it called “a precautionary and provisional action”; it said it had not determined that the model had definitively crossed the threshold.

The structure is conditional. If a model crosses a defined capability threshold, then a defined set of mitigations applies before it can be trained further or deployed. That conditional is what makes the commitment falsifiable: a reader, including one inside a regulator’s office, can in principle look at a model’s evaluations and the policy’s tripwire definitions and say whether the policy is being honored.

The choice of biosafety as the analogue was deliberate. The BSL classification is one strand of a broader pattern in which voluntary research-community guidance on dangerous biology eventually hardened into government rules. The recombinant-DNA moratorium that scientists imposed on themselves at Asilomar in 1975 is the most-cited precedent in the same lineage. The trajectory Amodei has been visibly betting on is the same one: a voluntary commitment that worked, was watched, was copied, and in time was codified.

The propagation

OpenAI’s Preparedness Framework, announced in December 2023 about six weeks after Bletchley, kept the basic shape: capability-triggered thresholds, pre-specified mitigations, an internal governance structure for deciding when a tripwire has been crossed. DeepMind’s Frontier Safety Framework, published May 2024, did the same. Three of the four most-watched Western frontier labs were now operating, on paper, under conditional commitments organized around the same logical structure. The fourth, xAI, was not.

This part of Amodei’s wager paid out. The 2023 conjecture was that competitors would adopt RSP-shaped frameworks once Anthropic demonstrated the commitments were commercially survivable, and competitors did. The “race to the top,” narrowly construed, ran. The second part of the wager has come under pressure.

Where each framework has loosened

Anthropic’s own amendment trajectory is the clearest record. The original RSP from September 2023 was followed by a v2.0 in October 2024 that added detail on ASL-3 mitigations, point releases v2.1 and v2.2 in 2025, a v3.0 in February 2026 that consolidated the policy and reframed parts of the binding structure, and further revisions through v3.4, effective July 8, 2026. Critics who post on the EA Forum and LessWrong, including people with prior Anthropic affiliations, argue that v3.0 softened explicit if-then commitments into language giving the lab more discretion about whether a tripwire has been crossed and what mitigation counts as adequate. The complaint predates v3.0. Of the October 2024 revision, a LessWrong commenter writing as aysja said the trigger was now set by “some opaque internal process at Anthropic that the document largely doesn’t describe.” The charge is the same in both rounds: commitments shifted from clauses an outsider could check toward judgments only Anthropic could make.

The v3.0 announcement contains a second admission, in Anthropic’s own voice, that does not map onto the critics’ register. The post says preset capability levels have turned out more ambiguous than the 2023 design implied; that in some cases models have approached thresholds without it being possible to determine whether they had definitively crossed; that the science of model evaluation is not yet mature enough to provide dispositive answers; and that the company has therefore taken a precautionary posture in cases where the evaluation data could not say. The 2023 prototype framing assumed that capability thresholds would be legible; that an outside reader, eventually a regulator, could read the model’s evaluations against the policy’s tripwires and answer the binary question. v3.0 conceded that the binary question often does not have a clean answer. That is a concession on how readable the framework is from outside, distinct from the question of whether discretion has expanded.

For OpenAI’s Preparedness Framework and DeepMind’s Frontier Safety Framework, the public record supports a weaker but parallel claim. Each has been amended at least once since adoption, and each amendment moved in the direction of more flexibility for the lab making the call. The same EA Forum and LessWrong critics read those amendments as analogous in shape to Anthropic’s. Less is publicly attested about exactly which clauses softened at OpenAI and DeepMind than is attested about Anthropic itself.

The picture across all three:

Source: each cell reflects the published framework version and critic readings on LessWrong and the EA Forum. Rows on what specifically softened are editorial synthesis from those critic readings; the more cautious cells for OpenAI and DeepMind reflect a thinner public record of clause-level commentary outside Anthropic’s own framework. All three RSP-shaped frameworks have been amended at least once since adoption, and in each case critics on LessWrong and the EA Forum read the amendment as loosening the binding edge.

What the libertarian right sees

The loudest opposition has come from the political right, where the argument is that voluntary safety commitments are not insufficient but insidious — a regulatory-capture play in the vocabulary of public interest. Marc Andreessen, the venture-capital cofounder of Andreessen Horowitz, has called AI regulation “the foundation of a new totalitarianism” and AI safety advocacy “a cult.” David Sacks, then the White House AI and Crypto Czar in the second Trump administration, accused Anthropic of running “a sophisticated regulatory capture strategy based on fear-mongering.” Yann LeCun, while Meta’s chief AI scientist, accused Amodei alongside Sam Altman and Demis Hassabis of “massive corporate lobbying… attempting to perform a regulatory capture of the AI industry.”

At a San Francisco conference in November 2024, answering Andreessen’s argument that AI is just math, Amodei asked: “Isn’t your brain just math?” Fortune ran the exchange.

The regulatory-capture critique is a motive critique. It says the firms advocating safety commitments stand to benefit if those commitments are codified, because compliance costs fall harder on smaller competitors. It does not require the safety case to be wrong, only that the firms making the case have an interest in winning it.

What Bengio sees

The harder critique comes from the safety side, and it predates the RSP framework by decades. Yoshua Bengio, the Université de Montréal professor who shared the 2018 Turing Award with Geoffrey Hinton and LeCun for foundational work on deep learning, has spent the last several years arguing that frontier AI needs an external regulator on the model of the Food and Drug Administration. In an April 17, 2026 essay for Fortune, Bengio wrote that “oversight of such high-stakes decisions cannot be left to private actors alone.”

The argument underneath that prescription is historical, not theoretical. The base rate on private actors policing themselves against catastrophic externalities, where the upside of cutting corners is large and the downside is borne outside the firm, is poor: a track record running through tobacco, asbestos, leaded gasoline, and the 2008 financial-stability frameworks the largest banks helped author. High-stakes self-regulation does not become reliable because the firm doing the regulating is well-intentioned. It becomes reliable, when it does, because external verification catches the cases where intentions and incentives diverge.

The structural problem follows. A capability-triggered framework is only as binding as the trigger evaluation, and the evaluation requires technical apparatus — red-teaming methodologies, capability benchmarks, weight-security audits, deployment monitoring — that no government currently has at the scale a frontier lab does. So the decisions about whether ASL-3 has been crossed, what mitigation is adequate, and whether a deployment is safe end up made inside the firm whose commercial incentives created the need for oversight in the first place. Under this reading, the prototype was never going to compile into regulation, because no regulator existed that could verify the trigger conditions. The voluntary framework is permanent by design.

Andreessen says the safety commitments are a stalking horse for capture. Bengio says they are sincere and still the wrong instrument, because the verification that would make them binding sits outside the firms that wrote them. The two critiques cannot both be right about the same firm at the same time; they target it from opposite directions.

The charitable readings of the loosening

The critics’ reading is not the only available one. Three more charitable readings sit alongside it.

The first is a pragmatic trade-off. Operating under the original binding if-then structure may have proved commercially unsustainable in ways not visible in 2023, particularly at the higher ASL rungs, where Amodei now describes the mitigations as requiring collective action. If Anthropic, the firm with the strongest reputational incentive to keep its own framework binding, has hit this wall, OpenAI and DeepMind plausibly hit similar walls earlier and chose softening over abandonment. The race partly worked: the frameworks still exist, even if the binding edge dulled.

The second is evolving empirical understanding. The original RSPs anchored their tripwires on capability thresholds that, with two more years of red-teaming and deployment data, may have turned out to be the wrong granularity. Amodei has said for years that capability progress is unpredictable in specifics; if so, the original tripwires were provisional, and a framework that did not update them would be the suspicious case. The opacity reading may be correctly observing the symptom — less explicit language — while misreading the cause as retreat from accountability rather than better calibration.

The third is coordination failure. Government regulation did not arrive on the timeline Amodei modeled in 2023, and in its absence the rational move for any frontier lab is to signal safety commitment to governments, hires, and enterprise customers without locking in commitments competitors who never adopted RSPs are not paying. Under this reading the firms that tried to hold the line were unilaterally bearing a cost the “race to the top” theory was supposed to socialize, and each adjusted independently when US regulatory follow-through did not arrive.

None of these require bad faith. All three are also consistent with Bengio’s structural critique: under any of them, the binding decisions still sit inside the firms, and the apparatus needed to test which reading is operative still does not exist outside them.

What Amodei has already conceded

Amodei has revised on the record three times. On Ezra Klein’s podcast in April 2024, he said “the level of political will and appetite for an assume-AI-is-dangerous-until-proven-safe attitude among policymakers” was lower than he had expected. The RSP v3.0 announcement in February 2026 went further on two counts: that mitigations at the higher ASL rungs are “outright impossible to implement without collective action,” and that preset capability levels have turned out more ambiguous than the original design implied, leaving the company in a precautionary posture in cases where the evaluation data cannot dispositively answer whether a tripwire has been crossed.

The first concedes that the regulatory follow-through the prototype was anchored on is not arriving on the timeline he initially modeled. The second concedes that the most demanding rungs of the ladder cannot be honored by any single firm acting unilaterally. The third concedes that the thresholds themselves are not as legible as the prototype framing required: a regulator could in principle read a model’s evaluations against the tripwires, but the evaluations and the tripwires do not yet meet cleanly enough for that read to be dispositive.

Where this leaves the argument

Amodei has said since Bletchley that “race to the top” is “about setting things up so that all of us can be the good guy.” On August 31, 2026, Anthropic went further and endorsed a lawful, verifiable, and effective mechanism for coordinated pacing among labs, a mechanism that does not yet exist. The framework history says that all three labs that adopted RSP-shaped commitments have, since adoption, given themselves more room for judgment about what being the good guy means in any specific case. Whether that room is the cost of keeping the framework alive at all, or evidence that voluntary commitment is the wrong instrument when the binding decisions have to be made by the firm being bound, is not yet resolved.

What is clear is the shape of the problem the prototype framing left behind. RSPs were pitched as drafts for a regulator, and in the United States none has arrived. The binding decisions still sit inside the firms whose commercial incentives created the need for outside oversight. The question Amodei has not publicly answered is what RSPs become if US regulation never arrives: he has said the prototype was not meant to be a substitute, and he has not said what the framework is in the absence of the codification that was supposed to follow.

Sources

• Anthropic, Responsible Scaling Policy v1.0 (September 2023), v2.0 (October 2024), v3.0 (February 2026), v3.4 (July 8, 2026), with the full version chronology — anthropic.com/responsible-scaling-policy
• Anthropic, statement on coordinated pacing (August 31, 2026) — anthropic.com/news/improving-alignment-security-efforts
• Anthropic, Activating AI Safety Level 3 protections (May 22, 2025) — anthropic.com/news/activating-asl3-protections
• European Commission, AI Act enforcement framework and obligations for general-purpose AI models — digital-strategy.ec.europa.eu/en/policies/enforcement-ai-act
• Office of the Governor of California, SB 53 signing announcement (September 29, 2025) — gov.ca.gov
• NIST, Center for Advancing Innovation and Standards for Super Intelligence — nist.gov/caissi
• Anthropic, UK AI Safety Summit prepared remarks (November 1, 2023) — anthropic.com/news/uk-ai-safety-summit
• OpenAI, Preparedness Framework (December 2023, with subsequent amendment)
• Google DeepMind, Frontier Safety Framework (May 2024, with subsequent amendment)
• Marc Andreessen, “Why AI Will Save the World” — a16z.com, June 6, 2023 (“AI risk” as “a cult”); on AI regulation as “the foundation of a new totalitarianism,” quoted by Fortune, November 21, 2024
• David Sacks, post on X — October 14, 2025, on Anthropic running “a sophisticated regulatory capture strategy based on fear-mongering”
• Yann LeCun, post on X — October 29, 2023, reported by CNBC, October 31, 2023, on “massive corporate lobbying… attempting to perform a regulatory capture of the AI industry”
• Dario Amodei, “Isn’t your brain just math?” — Fortune, November 21, 2024
• Yoshua Bengio, “Oversight of such high-stakes decisions cannot be left to private actors alone” — Fortune, April 17, 2026
• Ezra Klein podcast, Dario Amodei interview (April 2024)
• aysja, comment on Zac Hatfield-Dodds, “Anthropic’s updated Responsible Scaling Policy” — LessWrong, October 17, 2024 — lesswrong.com/posts/Q7caj7emnwWBxLECF?commentId=krnPsKmkaFLdFuQkQ
• LessWrong and EA Forum critic discussions of RSP v3.0 and analogous OpenAI/DeepMind framework amendments