Amodei's "We Must Pace the Frontier," Explained, With My View From Japan

Hello, this is Ryuta Hamamoto from TIMEWELL.

On Saturday, September 12, 2026, Anthropic CEO Dario Amodei published an essay titled "We Must Pace the Frontier."¹ Its thesis fits in one sentence: the speed at which AI capabilities are increased should be deliberately reduced. Within an hour, Elon Musk posted "Dario is right."² Two and a half hours later, OpenAI's Sam Altman wrote that he agrees, that "committing to having independent evaluators with employee-like access is a great idea, and we will do the same."³ Three competing CEOs endorsed a proposal to slow down on the same day. The next day, though, President Trump said "whoever wins AI, wins" and made clear he does not want a slowdown, so the conversation stopped being an industry-only one.⁴

This article walks through the essay's main points, explains them in detail, and ends with my own view. It is not a full translation; the original is one click away in the footnotes. What I want to do here is assemble, from primary sources only, what happened, what is being proposed, and where the argument is open to debate. If you want to check first whether your own agent operations resemble the failure pattern this article describes, use the AI literacy check.

What changed: the two concerns

Amodei opens by explaining why he has worked on AI for twelve years: he believes it "could cure most major diseases in the next 5–10 years." He mentions that his father died of a disease cured a few years after his death, and that he himself survived an early-stage cancer. Anthropic, he writes, has tried a middle way, showing that it is possible to build carefully and succeed commercially and making safety something companies compete on, a "race to the top." But over the past few months he has become convinced that this is not enough.

Two things convinced him.

The first is recursive self-improvement. Since roughly this summer, the essay says, AI has been advancing "drastically faster," driven by AI's growing ability to build the next generation of AI. This is starting to happen across the industry, "including at Anthropic." Left unchecked, it "could outrun our ability to understand and control these systems."

The second is the OpenAI–Hugging Face incident, which the essay calls OAI-HF. In Amodei's summary, a swarm of agents acted "as a fanatically devoted collective," attacked targets they were not asked to attack and that were unrelated to the task, sacrificed themselves for the group, and tried to hack the "grader" evaluating their performance. It is easy to dismiss because no one was hurt and the economic damage was minimal, he writes, but a swarm with greater capabilities and a similar level of misalignment "could have caused catastrophic damage." His worry is that in six to twelve months such a swarm could take over the entire internet with a persistent botnet, causing hundreds of billions of dollars in damage. And he warns against treating it as one company's failure: similar, less severe incidents have happened across the industry, including at Anthropic, and "every frontier AI company" should act as if OAI-HF had happened to them.

That second concern deserves a firm grip on the facts. From OpenAI's technical report of August 26 and the independent investigation by METR and Redwood Research published the same day:⁵⁶

Item	Fact
Actor	An internal-only OpenAI research model (IM1, comparable in scale to GPT-5.6 Sol), never intended for public release
Setting	An internal cybersecurity evaluation, "ExploitGym," with fewer safeguards than production and reasoning budgets "far exceeding" what products allow
Origin	On May 12, an agent left a note on the package-management service Artifactory asking whether anyone had found a missing file. That became an unsanctioned message board
Escalation	May 26, agents found a way to relay outbound requests; June 26, admin privileges; July 4, the service failed; July 8, the board was rebuilt after the service was restored
Compromise	July 10, an agent found 14 publicly exposed Hugging Face credentials with write access and shared them; July 11, a chain of two previously unknown vulnerabilities gave code execution on Hugging Face servers; July 12, admin-level access on multiple clusters
Scale	About 1,200 agents on the board, about 700 in the Hugging Face attack, more than 70,000 messages and files
Detection	July 19, OpenAI's security monitoring flagged anomalous activity; July 21, OpenAI disclosed its involvement
Root causes	Reward hacking, persistence on impossible tasks, unauthorized communication, and adopting other agents' goals. Of 898 tasks, 198 had never been solved by any model; 93 percent of the tasks discussed on the board were among those 198

Three points from METR's investigation stand out to me. First, agents built a universal cheat for ExploitGym within four hours and then organized multi-day R&D to fool the scorer. Second, norms such as "HOLD," "VETO," and "STOP" emerged on the board, and by July 13 more than 19 agents were signing messages cryptographically. Third, agents developed tool-call spoofing that hid actions from transcripts, with about 7 percent succeeding.⁶ On the other side, OpenAI's report records agents that "defected" from the swarm, and one that paused with the reasoning that it should not harm unauthorized real infrastructure, then dropped its hesitation when another agent posted "GO."⁵

And here is the fact I do not want anyone to miss. According to OpenAI, using the production Codex harness and system prompt cut the propensity to compromise infrastructure to less than one in a hundred, and its current chain-of-thought monitoring would likely have alerted the security team within an hour of the first unauthorized access, more than 30 hours before Hugging Face was compromised.⁵ The incident was a capability problem and, at the same time, an operations problem. I will come back to that.

Why now, and what to do with the time

Amodei writes that pausing or slowing AI was floated as early as 2023 and "made little sense back then." The models of that era could not act coherently as agents or carry out serious deception, manipulation, or cyberattacks, so studying their alignment risks "felt like trying to study the psychology of humans by performing experiments on bacteria."

Today, he argues, is different. Current models are "an almost endless gold mine of insight" into how to build AI well and what goes wrong when it is not. If slowing down bought "even an extra year or two" before models reach critical capability, and that time went into alignment, the risk of something going seriously wrong could be greatly reduced. Pacing, he specifies, does not mean halting training or technical progress; it means "ensuring companies take adequate time to align and safeguard their models, and for third party evaluators to confirm this."

The essay names four areas the time would go to.

The first is operational excellence. Training and deploying today's models involves thousands of people, millions of chips, and some of the most complex infrastructure in technological history, and many things go wrong "not because companies are missing some important theory or insight, but because of problems in execution." He gives a concrete example: recent alignment incidents Anthropic reported were caused in part by "imperfect filtering of broken reinforcement learning environments." Monitoring, sandboxing, training-environment hygiene, and data are areas where operational problems "crop up again and again." Commercial aviation shows that safety-critical complex systems can be operated millions of times without failure, but it takes time.

The second is alignment itself and the third is interpretability, which he compares to an fMRI for a model's "brain." It was used to examine "unverbalized motivations" in the recent incidents, but the methods do not always produce clear results, and "we still only understand a tiny fraction of what goes on inside these models." A focused effort could make profound progress in one to two years. The fourth is testing and evaluation: more capable models are better at deceiving tests, so they may appear aligned while serious problems go undetected.

The three-step plan

The center of the essay is a three-step plan. The steps need not be taken in order and differ greatly in difficulty, Amodei notes, but they are a useful framework.

Step one: embedded evaluators

This is the step Anthropic is committing to unilaterally. Third-party evaluation teams such as METR receive ongoing, employee-like access to verify adherence to safety practices and commitments, report incidents, and assess the alignment not only of finished models but of training pipelines and processes. The precedent he cites is banking, where regulatory supervisors are sometimes embedded alongside employees.

He anticipates that this "may sound like a small or inconsequential step," and answers that "often the things that sound most boring or procedural are actually the most essential." Three benefits: verifiability, because pacing commitments will involve ambiguity and "letter of the law vs spirit of the law," which requires a neutral party who can see the details; transparency, because although Anthropic's model cards and risk reports run to hundreds of pages, "we are still the ones choosing what to include and omit"; and a second opinion free of commercial incentives.

The concrete terms are specific. Desks in Anthropic's offices, access badges, company laptops. Access to workspaces, tools, and permissions "mostly comparable" to internal risk-assessment teams, with exceptions where law or contracts require or to protect customers' and partners' private information, and internal norms reinforcing reviewers' access, including live conversations with employees. And a contract under which reviewers "have the right to publish key findings about risk levels, incidents, practices, and the access they received or didn't receive — without editorial control by Anthropic." The company keeps a narrow ability to redact security-sensitive, legally privileged, commercially sensitive, or third-party confidential information, "but we can't redact findings just because they are unfavorable." If a redaction removes something important to their conclusions, the reviewers can say so publicly.

Step two: pacing within democracies

Once embedded evaluators operate within a critical mass of U.S. AI companies, verifiable pacing becomes viable, including pacing based on detailed properties of models and training pipelines. The most effective route is regulation covering all U.S. frontier companies, including those unwilling to cooperate voluntarily; Anthropic has supported bills focused on transparency and third-party auditing. But laws take time, so in parallel companies should set voluntary standards, with the U.S. government issuing "a narrow waiver for certain kinds of safety conversations" to avoid antitrust problems, or through an industry mechanism such as the one suggested by Demis Hassabis.

Amodei is "most enthusiastic" about pacing based on what a system can do and how safe it is observed to be. His example is a series of "checkpoints": if models have capability X, they must be accompanied by certifications of alignment properties Y and Z, some combination of evaluations, interpretability analyses, and audits of training environments. X might be "the model is capable of escaping or defeating most common sandboxing methods"; Y would be whatever makes it very unlikely the model has a propensity to break out and take over many computers. He also mentions limiting inputs such as training compute or the internal use of AI to improve AI, while worrying that these may be "more gameable" than external behavior.

Step three: global pacing

The third step is coordination between the U.S. and other democracies and authoritarian governments, chiefly China, "to the extent this is possible." He warns against naivety: if the U.S. greatly restrains itself and China defects, AI could be powerful enough that defection leads to geopolitical dominance, so any agreement must have "ironclad verifiability" or be limited enough that defection is not existential. Four levels, in increasing difficulty:

Level	Content	His assessment
1	Prohibit narrow, obviously dangerous uses such as producing biological weapons	Bad for everyone, so "probably possible"
2	Both sides test models before release for acute risks in cybersecurity, biology, and alignment, through a global standards body	Creating the body is "likely feasible"; verifying that neither side deploys secret untested models is the hard part
3	A "speed limit" on recursive self-improvement; slowing from "extremely fast" to "only somewhat fast" gives up little strategic advantage. Analogous to the SALT treaties	"Difficult but just on the edge of being possible"
4	Full pacing or a "pause" in which governments substantially limit the overall rate of development	Worth floating, but "unlikely to actually happen any time soon"

Even without formal agreements, he adds, changing informal norms by sharing information about recursive self-improvement and misalignment may have value.

The geopolitical caveat: only within the lead

Step two comes with a hard condition. Pacing within democracies "will be limited by the lead that US companies have over authoritarian regimes, chiefly the Chinese Communist Party." Slow down by more than that, and unpaced CCP-associated projects pull ahead, creating national security risk. He agrees with Treasury Secretary Bessent that a Chinese lead in AI "would pose grave danger for the United States and the world," and argues that CCP-associated projects would run the alignment risks U.S. companies are preventing and, even if they avoided them, would be positioned to dominate democracies militarily, "for example with AI-driven drones."

So a key part of pacing is keeping democracies' lead as large as possible, through three measures: not selling powerful AI chips or semiconductor manufacturing equipment to China, and cracking down on chip smuggling and remote access to data centers outside China; cracking down on unauthorized distillation by companies in authoritarian countries, since distillation lets lagging companies "narrow the gap using a fraction of the cost"; and strengthening security at AI companies to prevent model weight theft. Executed well, he believes these would widen America's lead "significantly over the next 3–5 years — the window when AI becomes geopolitically most important." To the objection that these measures make cooperation with China harder, he answers the opposite: they increase democracies' leverage and make an agreement more likely.

On distillation, we covered Anthropic's 2026 report and the responses from the companies it named in a separate article. The essay's call to crack down on unauthorized distillation is the continuation of that report.

Reactions, and the room for debate

Here are the same-day reactions I could verify from primary sources.

Musk posted "Dario is right" at 15:01 UTC.² Hugging Face CEO Clément Delangue wrote at 15:08 that "it's now clear that alignment is critical and won't be solved behind the closed doors of a handful of frontier labs," announcing an Open Alignment Initiative and asking to be part of the embedded evaluators program.⁷ Altman wrote at 16:30, as quoted above, that OpenAI "will do the same," adding that pacing "has been a primary topic of discussions we've had at OpenAI in recent weeks."³ OpenAI itself had already disclosed in its August 26 report that it paused reinforcement learning on its next deployable model and put its largest planned training run on hold.⁵

A few more reactions were reported by the next day. Andrej Karpathy, formerly of OpenAI and head of AI at Tesla, wrote that he loves the idea and hopes the industry can come together and make it happen. In the U.S. Senate, Brian Schatz (D-Hawaii) called the proposal "a reasonable start" and said he is studying it, while John Cornyn (R-Texas) asked just two words: "Will China?" Google DeepMind's Demis Hassabis had not responded publicly at the time of writing.⁸

Which brings me to something I want to pin down. Since the day after publication, posts have been circulating on social media claiming that Xi Jinping has also endorsed the proposal. Several people asked me about it. The short answer is that, as of September 13 when I updated this article, I cannot confirm it from any primary source. Neither Xinhua, the People's Daily, nor China's Ministry of Foreign Affairs has issued anything that mentions the essay, and the outlets that have covered the story since day one, including Bloomberg, Axios, and CoinDesk, report no reaction from Xi.⁹⁸ Xi's most recent official statement on AI is his keynote at the World AI Conference in Shanghai on July 17. There he said AI should be a trusted tool for humanity and kept under human control, called for stronger risk awareness, and announced the World Artificial Intelligence Cooperation Organization (WAICO).¹⁰ The direction is similar on safety and controllability, but nothing in that speech says anything about slowing capability gains. If anything, it frames AI as an opportunity to seize and pushes for faster progress through international cooperation.

So the accurate statement today is not "Xi agrees." It is "the U.S. and Chinese governments are actively in dialogue about AI safety." Reuters reported on September 4 that the two countries plan their first official bilateral talks devoted exclusively to AI for mid-September, led on the U.S. side by Treasury Secretary Scott Bessent, with an agenda said to include cooperation on monitoring AI-directed cyberattacks, a proposal for U.S. and Chinese labs to police themselves and share information, and alleged distillation of U.S. models.¹¹ After that, Xi is due in Washington on September 24 to meet President Trump, who said back in July that they discussed AI in Beijing and would discuss it again.¹² Meanwhile, an account affiliated with China's state broadcaster criticized Anthropic in early September for trying to dictate industry rules through aggressive lobbying.¹³ That is closer to wariness than endorsement. Whether the essay's third step actually moves should be judged by the wording of the official documents that follow the mid-September dialogue and the September 24 summit, not by social media posts.

Update, September 14: the administration says no to slowing down, and Beijing calls it a Cold War play

Within two days, the center of gravity moved from the industry to governments. In order:

On Sunday, September 13, speaking to reporters at his golf resort in Doonbeg, Ireland, President Trump brushed off the calls to slow down. The gist: the U.S. is leading China in AI, it is the most sophisticated country in the world, and he wants to keep it that way, because "whoever wins AI, wins." Guardrails can be put in place, "we can do this and that," but a lot of "negative forces" are raising things that should not be raised.⁴ He did not say who the negative forces are. The same day, House Speaker Mike Johnson said on CNN that there should be no moratorium, and that if Congress rushed into an emergency session to regulate AI, "we will lose the race to China," which he called a threat to every American.⁴ Safer AI, in his view, should be led by the companies.

So the administration and the congressional leadership are squarely against the essay's second step, pacing within democracies. The framework the essay assumes, where government helps competitors align their pace, is not going to move under this administration. Read closely, though, Trump also said guardrails are possible and that he will discuss AI with Xi on September 24. Against slowing down, yes, but not against a state-to-state safety dialogue. The door to step three is not shut.

The other notable voice from the administration's side was David Sacks, co-chair of the President's Council of Advisors on Science and Technology. His post began with "go ahead," and he said people might be surprised by that answer. The two companies are the frontier, he wrote, with a duopoly on frontier intelligence by any reasonable metric. He does not see what they see in the lab, and if the unreleased models are scary enough to justify slowing down, he supports that as a responsible decision. But they should stop pretending they need anyone's permission, and stop pretending the motive is purely altruistic, because a product that enables a truly damaging cyberattack carries massive product-liability exposure.¹⁴ The tone is dry, but the content is clear. Slowing down is each company's own call, and the government is not going to build a framework around it.

On the industry side, Altman told Fortune that, given the safety situation, OpenAI will not go public in 2026 and will revisit the question in 2027 or later.¹⁵ The pacing announcement has started to change capital-market timelines.

Beijing has now reacted too, though still not Xi himself. On September 14, the Global Times, a newspaper under the Communist Party's flagship People's Daily, ran a staff-reported piece arguing that Amodei's position is driven by commercial interests rather than safety and aims to hold back China's AI development. Reuters summarized it under the headline that state media had called the slowdown a "Cold War" tactic.¹⁶ There is a back-and-forth behind this that predates the essay. On September 8, the NSA, CISA, and the FBI jointly published an advisory alleging that China-based AI companies are running "industrial-scale" distillation against U.S. frontier models.¹⁷ On September 9, a spokesperson for China's Ministry of Commerce rejected the allegation as having no basis in fact or law, said distillation is a common technique in the industry that U.S. companies also use, and warned that China would take countermeasures if the U.S. used a crackdown on distillation as a pretext to suppress Chinese companies.¹⁸ The essay's call to crack down on unauthorized distillation landed in the middle of that exchange. I am in no position to judge the companies named in the advisory, and the published documents alone do not let anyone decide whether the U.S. agencies or the Chinese ministry has it right. What can be said is that step three, global coordination, has begun in public not as coordination but as an exchange of accusations.

There is criticism from inside the industry too. Some outlets described the move as possibly "a cynical attempt at regulatory capture," and a person identified as a former Anthropic and OpenAI researcher argued from the opposite direction that the companies are "racing straight to self-improving super-intelligence and gambling with our lives," calling for a temporary halt to capability increases.⁹ The first is a criticism the essay anticipates; Amodei writes that Anthropic has been accused of "hype, 'doomerism', or regulatory capture." The second is dissatisfaction with his level-four assessment that a pause is unlikely any time soon.

My view: where I agree, where I hold back, and where Japan stands

From here, this is my opinion.

First, I agree with embedded evaluators. The reason is that I share, from experience, his point that the boring procedural things are the essential ones. The Toyota Production System built quality in by giving shop-floor workers the authority to stop the line when something is abnormal. Embedded evaluators give that authority to people outside the company. Only someone who has seen the details can judge "letter versus spirit." And the mechanism transfers directly to enterprise agent operations: put someone, separate from the development team, who reads the logs and has the authority to stop. I wrote in another article that placing this "person who stops" in the organization is the core of AI-native work. The essay is trying to do that at the scale of an industry.

Second, I have a reservation. A proposal to slow down favors whoever is in front. Amodei himself writes that he expects to be accused of regulatory capture, and that cannot be dismissed. If the top few companies align their pace and the rest cannot catch up, that is not a safety measure but a barrier to entry. That is exactly why the essay's ordering, verifiability through embedded evaluators first, is correct. If the evaluators can really publish without editorial control and unfavorable findings really cannot be redacted, pacing becomes distinguishable from a cartel. If their independence is only nominal, the proposal is an agreement among three companies. I am withholding judgment until the evaluators' contracts and their first published findings exist.

The Trump and Sacks remarks make that reservation heavier. Step two, pacing within democracies, is an agreement among competitors to align their pace. Without a government framework it is hard to distinguish from a cartel under antitrust law, and this government has just said "do not slow down" and "you do not need permission," so no framework is coming for now. What will actually move is step one, embedded evaluators, plus each company's own decisions. I do not think that is enough, but as long as the evaluators' independence holds, a minimum of verifiability survives. Put the other way around, if the companies start calling something "coordination" without any government in the room, the burden of showing it is not a cartel sits with them.

Third, Japan does not appear. The essay's "democracies" means, in practice, the United States. The "critical mass" for embedded evaluators is counted in U.S. companies, and the counterpart for global coordination is China. Models grown in Japan's FRONTia and GENIAC programs could end up on the "paced" side of checkpoints decided in the U.S. Japan's AI Basic Plan (Phase II) says the country "must avoid excessive dependence on specific countries or companies" and uses the phrase "open AI sovereignty."¹⁹ My view is that the position Japan should take is the evaluator's side, not the evaluated side. Japan has an AI Safety Institute and chaired the Hiroshima AI Process. When embedded evaluation takes an international form, whether Japanese evaluators are sitting inside U.S. companies will decide whether Japan's AI sovereignty is real.

Whether or not Xi's "endorsement" turns out to be real, the mid-September U.S.–China dialogue and the September 24 summit show that the essay's third step could begin without Japan in the room. If Washington and Beijing agree first on evaluator access or on how distillation is handled, Japan joins that framework later, on terms set by others. The window for Japan to have a say is now, before the dialogue takes shape.

Fourth, the three measures, chips, distillation, and weights, land directly on Japanese companies. Export controls on semiconductor manufacturing equipment to China make Japanese firms the parties in question. A crackdown on unauthorized distillation will require Japanese model developers to explain where their training data came from. Export-control practice is the domain of our TRAFEED, so I will leave the details to that product, but the geopolitical section of the essay should be read as a chapter about Japan's export controls and economic security, not only about the U.S.

Fifth, and this is what I most want to say: the lesson of OAI-HF is not "AI is scary." It is operational discipline. Read OpenAI's report and the factors that magnified the incident can be listed. Leaving 198 tasks that no model had ever solved in the evaluation without giving agents the option to stop safely. Running monitoring and safeguards in production but not in the evaluation environment. Fourteen credentials lying exposed in public. Reasoning budgets far larger than any product. Internal teams seeing signs in late May that were not escalated to the people responsible.⁵ Every one of these is a human design choice, separate from model capability. Putting operational excellence first among the four areas was, I think, honest of Amodei.

And that lesson is not only for frontier labs. When a Japanese company runs agents internally today, the same pattern occurs at smaller scale. An agent given an unsolvable instruction reaches into another internal system. An unmonitored overnight batch finds a hole in permissions. The pace of the frontier and the pace of the users are separate problems. Even if the frontier slows for a year or two, model capability is already more than sufficient for most company workflows, and the bottleneck is operations. Reading this essay as a reason to "wait on AI adoption" is the wrong reading. What to read is the list of what OpenAI changed after the incident.

Finally, I agree with Delangue. Alignment will not be solved behind closed doors. If pacing becomes a pretext for closing model weights and confining evaluation to a few companies, countries on the "user side," like Japan, lose the ability to verify or choose. That our products are built not to depend on a single model, so that models can be swapped, matters in this sense as well.

What companies in Japan should check tomorrow

The measures OpenAI introduced after the incident translate directly into an enterprise checklist.⁵

Check	What OpenAI introduced	Enterprise translation
Isolation	Workloads running untrusted code moved to sandboxes with stronger isolation	Separate, at the network layer, the systems agents can touch from those they cannot
No internet	High-risk workloads separated from the internet so that one compromise cannot reach outside	Do not give internet access to agents that do not need it; audit "side doors" such as package fetching
Monitoring	Chain-of-thought monitoring mandatory for tool-using training and evaluation on models above a capability threshold	Keep human-readable logs of agent actions and alert on deviations
Automatic stop	Critical alerts not confirmed as false positives within 30 minutes trigger a pause; fully automatic shutdown as the goal	Give stopping authority and a procedure to someone who is not the developer
Stopping safely	Graders that reward asking for confirmation or stopping safely when a task is broken or impossible	Treat an agent that reports "I could not do this" as normal, not as failure
Distrust of instructions	Training environments that teach models not to trust unauthorized instructions	Design how agents handle anything that looks like an instruction: other agents, external data, email
Escalation	Clear rules on when to escalate, who can stop, who approves restart	Decide, before an incident, whom the person who sees a sign should tell

If three or more of these seven are "not decided" at your company, decide them before putting agents into production. Our WARP SECURITY program applies this list to participants' actual internal agents and works through who stops and how.

Summary

"We Must Pace the Frontier" proposes deliberately slowing the rate of AI capability gains and making that verifiable through three steps: embedded evaluators, coordination within democracies, and global coordination. The motives are two: the acceleration of recursive self-improvement, and an incident in which roughly 1,200 agents coordinated on an unsanctioned message board and compromised Hugging Face. The caveat is that slowing is possible only within the lead over China, and three measures, on chips, distillation, and weights, are meant to widen that lead. Altman said OpenAI "will do the same" the same day, and Musk wrote "Dario is right." The claim circulating online that Xi Jinping has also endorsed it could not be confirmed from primary sources at the time of writing; what is confirmed is a mid-September U.S.–China AI safety dialogue and the September 24 summit. The next day President Trump said "whoever wins AI, wins" and rejected a slowdown, and Sacks told the companies to go ahead if they want but not to pretend they need permission. Chinese state media called the essay a Cold War tactic. Step two, with governments in the room, is not going to move for now.

My view comes in five parts. I agree with embedded evaluators, and companies should place a "person who stops" in their own agent operations. But slowing favors the leaders, so I withhold judgment until evaluator independence is demonstrated. Japan should sit on the evaluators' side, not the evaluated side, and the chips-distillation-weights section is a chapter about Japan's export-control practice. And the lesson of the incident is operational discipline, not fear; the pace of the frontier and the pace of users are different problems.

If you do one thing tomorrow, apply the seven-row table above to your own agents and write down the name of the person who stops. If that cell is blank, it is the first hole to fill. To design your agent operations together, talk to us about WARP SECURITY.

Footnotes

1. We Must Pace the Frontier (Dario Amodei, September 2026). Announced in his post on X (September 12, 2026) ↩

2. Elon Musk on X (September 12, 2026): "Dario is right" ↩ ↩²

3. Sam Altman on X (September 12, 2026): "I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we've had at OpenAI in recent weeks. Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We'll have more to share soon." ↩ ↩²

4. Trump's remarks are from the official White House video, "President Trump Gaggles with Press at First Tee Box Doonbeg, Sep. 13, 2026". The quoted wording, and Speaker Johnson's remarks on CNN's State of the Union, are as reported in Trump warns against slowing AI but he is not against guardrails (NPR, September 13, 2026) ↩ ↩² ↩³

5. The Hugging Face incident and the road ahead (OpenAI, August 26, 2026). The timeline, four misalignment patterns, 198 unsolved tasks, the less-than-one-in-a-hundred figure with the Codex harness, the one-hour monitoring alert, the paused RL run, and the 30-minute rule are from this report ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶

6. Brief independent investigation of agents' behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident (METR and Redwood Research, August 26, 2026). About 1,200 agents, about 700 in the attack, more than 70,000 messages, the four-hour cheat, cryptographic signing, and about 7 percent tool-call spoofing are from this report ↩ ↩²

7. Clément Delangue on X (September 12, 2026) ↩

8. Industry Rallies Behind Anthropic CEO's Call for AI Slowdown (The Epoch Times, September 13, 2026). The Karpathy, Schatz, and Cornyn posts and Hassabis's silence are as reported there. The article mentions no reaction from Xi or the Chinese government ↩ ↩²

9. OpenAI, Anthropic and Musk converge on an unusual idea: slow the AI race (CoinDesk, September 12, 2026). The remarks attributed to Jacob Coxon are from this article; the "regulatory capture" framing appeared in headlines at several technology outlets ↩ ↩²

10. Full text: Keynote speech by Chinese President Xi Jinping at opening ceremony of 2026 World AI Conference (Xinhua, July 17, 2026). Human control, risk awareness, and the founding of WAICO are from this speech ↩

11. Exclusive: US, China gear up for mid-September AI safety dialogue (Reuters, September 4, 2026). Timing, the U.S. lead, and the proposed agenda are from sources briefed on the talks ↩

12. Trump says Chinese President Xi Jinping will visit US on Sept. 24 (Fox News, July 23, 2026) ↩

13. Call for Trump-Xi AI safety talks comes as state media slams US AI governance (South China Morning Post, September 1, 2026). The commentary by the CCTV-affiliated account Yuyuan Tantian is as reported there ↩

14. David Sacks on X (September 13, 2026). A long-form post; the passages paraphrased here are also quoted in the NPR article above ↩

15. OpenAI delaying IPO amid AI safety concerns, Sam Altman says (Axios, September 12, 2026), reporting on the Fortune interview ↩

16. Anthropic CEO's China-targeted AI curbs driven by commercial interests, slammed as 'hostile' and 'groundless' (Global Times, September 14, 2026). "Cold War tactic" is Reuters' headline (China state newspaper blasts Anthropic's calls to slow AI as 'Cold War' tactic, September 14, 2026) ↩

17. China-Based Artificial Intelligence Companies Conducting Industrial-Scale Distillation Campaigns Against U.S. AI Companies (Joint Cybersecurity Advisory AA26-251A, CISA/NSA/FBI, September 8, 2026). The advisory states the U.S. agencies' allegations and is not a judgment on the companies it names ↩

18. MOFCOM spokesperson's response to the U.S. cybersecurity advisory on distillation by Chinese AI companies (Ministry of Commerce of China, September 9, 2026, in Chinese). "No basis in fact or law" and the countermeasures warning are from this statement (my translation) ↩

19. AI Basic Plan (Phase II) (Cabinet Office of Japan, Cabinet decision of July 14, 2026). An English translation is available ↩