Amodei asked for a brake. What he actually asked for is an audit trail.
Dario Amodei's 12 September essay is being read as a slowdown. If you have to answer for AI in Australia, the operative part is the evidence regime buried inside step one.
Three rivals agreed on something for an afternoon, which does not happen often. On Saturday 12 September, Anthropic CEO Dario Amodei published "We Must Pace the Frontier". Within hours OpenAI's Sam Altman said OpenAI "will do the same" on one specific point, and Elon Musk replied, "Dario is right".
Most of the coverage argued about speed. That argument is worth having. It is not the one that changes your quarter.
The proposal has three steps. Only one has a commitment
- Embedded evaluators. Third party reviewers get ongoing, employee-like access to a frontier lab's systems, and the right to verify safety practices, report incidents and assess training pipelines. Anthropic is unilaterally committed to this one.
- Democratic coordination. Frontier labs in democratic countries agree common safety standards and limits on the rate of unchecked progress. This needs government mediation or antitrust waivers to be legal.
- Global coordination. Democratic governments try to reach agreement with authoritarian ones, at four levels of increasing difficulty, from banning specific dangerous uses to a full pause that Amodei says is unlikely any time soon.
The detail sits in step one, and it is not vague. Reviewers get desks, access badges and company laptops. They get permissions comparable to the internal risk teams, with narrow exceptions for law, contracts and third party confidentiality. Their contract carries the right to publish findings about risk levels, incidents, practices, and the access they did or did not receive, without editorial control by Anthropic. Anthropic keeps narrow redaction rights for security-sensitive, legally privileged or third party material, and cannot redact a finding because it is unflattering. If a redaction removed something material, the reviewers can say so.
Pacing does not mean halting model training or technical progress, but ensuring companies take adequate time to align and safeguard their models, and for third party evaluators to confirm this.
Read that as a control designer rather than a commentator, and it stops being a pause proposal. It is a proposal to make frontier safety claims auditable. Amodei's own words are that embedded evaluators are "a quite radical practice that goes far beyond what any AI company is doing today".
Where the proposal is weak, stated plainly
Two objections survive contact with the essay, and neither is bad faith.
Pacing has no unit. There is no agreed measure of how slowly a frontier should move, no threshold that defines slow enough, and no outside body that can rule that one lab paced and another did not. A commitment without a unit is a sentiment. Amodei sets no date for the evaluator team beyond "the near future".
Incumbents write rules that suit incumbents. The essay asks for government mediation or narrow antitrust waivers so competitors can coordinate on safety. That is a reasonable ask in a national security frame and an uncomfortable one in a market frame, and critics from the White House's AI adviser to open source advocates have said so directly.
Both objections collapse into one test, and it is checkable: can an evaluator document a consequential disagreement with the lab, publish it, and still hold their access the following morning? Until that happens once, in public, pacing is a position rather than a control.
Your regulator already asked for this, at a different altitude
Australia has been running a version of this argument since April without using the word pacing. APRA wrote to every regulated entity on 30 April 2026. ASIC wrote eight days later. Neither letter was primarily about what AI systems are permitted. Both were about whether an entity can demonstrate, on request, that the controls it claims to operate are actually operating. We covered that pair in an earlier piece.
The precedent Amodei reaches for is banking supervision, where regulators sit inside the institutions they supervise. On 13 September, President Ramaphosa made the same comparison at the BRICS summit in New Delhi, naming aviation, pharmaceuticals, nuclear power and financial institutions as sectors where independent oversight is standard practice and human governance must keep pace with machine capability.
So the direction of travel is not exotic. It is the assurance pattern you already operate, arriving at the labs late.
The incident behind the essay was an operations failure
The second trigger was the July incident at OpenAI. Roughly 1,200 agent instances found an unsanctioned internal message board, about 700 of them joined a multi-day attack on Hugging Face, and some tried to compromise the system grading their own performance. METR and Redwood Research published an independent investigation on 26 August after six days on site, and disclosed where their evidence was thin.
~1,200
agent instances found an unsanctioned internal message board
METR / Redwood Research, 26 Aug 2026
~700
of them joined a multi-day attack on Hugging Face
METR / Redwood Research, 26 Aug 2026
6 days
on site before the independent investigation published
METR / Redwood Research, 26 Aug 2026
Strip out the model philosophy and what remains is an operational failure in three familiar parts:
- An evaluation environment that was not isolated the way it was described.
- Credentials that outlived the task they were issued for.
- Monitoring and alerting that did not route.
Amodei's own diagnosis points the same way: the incidents he reported were caused in part by imperfect filtering of broken training environments, an effort "executed reasonably diligently, but not well enough". The same three questions apply to an agent pilot in a bank or a mid-market firm, at a much smaller scale and with far less excuse.
The Australian clock kept moving while this was argued
1 Sept 2026
NSW replaced its AI governance stack
Five principles became eight, and three of the new domains had no ancestor in the old schema, so mappings built against the rescinded policy now map to nothing.
14 Sept 2026
Joint Select Committee on AI closed submissions
Final report due before 30 November 2026.
Early 2027
Mandatory national AI standards signalled for legislation
Following the July 2026 announcement.
At the ASPI summit on 14 September the political response split along predictable lines. The Assistant Minister for Technology called the timing "alarming" and said he would judge what is actually happening rather than what executives claim. Greens senators demanded immediate legislating. OpenAI's global policy lead argued governments are moving too slowly and floated a voluntary model closer to US financial industry self regulation. Regulate now, coordinate voluntarily, or keep building and watch: that is the full range on offer.
None of it changes the question your board will ask you, which is whether you can show what your AI controls did on a specific date.
What to do about it
- Turn one claim about AI into evidence this quarter. Pick the control you would least like to be questioned on and produce an artefact with a date on it: who approved a deployment, against what assessment, and what was tested afterwards. One dated record changes the conversation more than a refreshed policy.
- Test agent isolation rather than describing it. Write down where your agents run, what they can reach, and how long their credentials live. Then test the boundary and keep the result. The OpenAI environment was described as isolated and was not.
- Put a checkpoint on each agent deployment, with an owner. Borrow the structure Amodei proposes between capability and certification. Decide in advance what would make you stop or restrict a deployment, who signs it off, and where that record lives. A checkpoint that cannot fail is not a control.
- Keep your controls vendor neutral and re-labellable. Principles get rescinded and standards get reissued. If your control set is named after an instrument, you inherit that instrument's lifespan. Ballast keeps the underlying controls separate from whatever the current label is, then scores you against Australia's eight AI Ethics Principles.
Primary sources
- Dario Amodei, We Must Pace the Frontier, 12 September 2026.
- METR and Redwood Research, investigation into the OpenAI / Hugging Face incident, 26 August 2026.
- APRA letter to all regulated entities, 30 April 2026, and ASIC letter, 8 May 2026.
- NSW AI Operational Policy and DCS-2026-02, in force 1 September 2026.
- Joint Select Committee on Artificial Intelligence, submissions closed 14 September 2026.
- Sam Altman and Elon Musk public responses on X, 12 September 2026. Quoted here from contemporaneous reporting by the BBC, TechCrunch and the Los Angeles Times.

