The optimistic reading of last week’s events goes something like this: the world’s most powerful AI executives — Anthropic’s Dario Amodei, OpenAI’s Sam Altman, and SpaceX’s Elon Musk — finally broke ranks with the hype machine and publicly called for a deliberate slowdown in AI development. Surely, the thinking goes, the adults are in the room.
The less comfortable reading is that their intervention arrived at exactly the moment it was least likely to work — and may have made cooperative AI governance harder, not easier.
Within 48 hours of Amodei’s widely read essay calling on companies to “slow the pace” of model improvement, both Washington and Beijing had firmly rejected the premise. Not on the same grounds, and not in agreement with each other — but in a striking display of parallel dismissal that should alarm anyone who believes international coordination on AI risk is achievable in the near term.
The Reading
What Actually Happened This Week
On Saturday, Amodei published a lengthy essay calling for a deliberate industry-wide reduction in the pace of AI model development, citing mounting safety concerns. The essay was publicly endorsed by Altman on X and carries the implicit weight of Musk’s alignment given his prior warnings. Amodei also argued that a “Chinese lead in AI would pose grave danger for the United States and the world,” and called for restricting U.S. technology exports to China — a framing that did significant geopolitical work alongside its safety argument.
By Monday, Beijing’s Foreign Ministry had a terse reply. Spokesperson Guo Jiakun told reporters that “fearmongering, confrontation and malicious competition will only disrupt the process of global AI governance and serve no one’s interests.” The state-run Global Times was blunter still, calling Amodei’s essay a “‘Cold War playbook’ for the AI sector” whose slowdown provisions were fundamentally aimed at containing China rather than protecting humanity.
President Trump, meanwhile, took his own pass on the safety warnings — from a different direction. “We’re leading China in AI,” he told reporters Sunday. “Whoever wins AI wins.” He dismissed the concerns as coming from “very negative forces” raising things “that won’t happen.” For more on how Trump has consistently sidelined industry safety voices, see our earlier coverage of Trump dismissing AI slowdown calls as Amodei, Musk, and Altman sounded the alarm.
The result: a week that began with a rare show of executive humility about AI risk ended with both superpowers signaling, loudly, that they are not interested in slowing down.
The Overlooked Risks in the Optimistic Framing
The optimistic case for Amodei’s essay rests on a belief that naming a risk publicly creates pressure to address it. But that logic assumes the actors with the most power to respond share enough common ground to act. This week’s events suggest they do not.
The deeper problem is structural. Amodei’s call for restraint and his call for export controls on China are not easily separable — they appear in the same document, making it almost impossible for Beijing to engage with the safety argument without implicitly accepting the framing that China is the threat. Beijing read it that way, and the reading is not entirely unreasonable. As Ja Ian Chong, an associate professor of political science at the National University of Singapore and a nonresident scholar at Carnegie China, put it: “There’s talk of cooperation, but the behavior suggests otherwise — underlying that is deep mutual suspicion.”
On the U.S. side, the risks of the current approach are equally underweighted. Trump’s response — that America simply needs to win — is strategically coherent by his own lights, but it forecloses the kind of bilateral frameworks that could establish even minimal safety floors. Altman himself acknowledged on X that international coordination would require government help. That help appears unlikely under an administration whose primary metric is competitive dominance. The broader tension between the AI industry and the political environment it operates in has been explored in our analysis of why the AI slowdown debate is more complicated than it looks.
Meanwhile, AI models have continued to advance faster than the technical safeguards designed to constrain them — a pattern that has now been running for several model generations. Separate from the geopolitical noise, that underlying dynamic has not changed.
Historical Parallels Worth Taking Seriously
Amodei himself reached for the Cold War arms race as a metaphor in a CBS interview, saying the two sides should work together but that he didn’t “hold out much hope.” The comparison is instructive — but perhaps not in the way he intended.
The nuclear arms race did eventually produce arms control agreements: the Limited Test Ban Treaty, SALT I, the Non-Proliferation Treaty. But those agreements took decades, required catastrophic near-misses — most notably the Cuban Missile Crisis — and still left both sides with arsenals capable of mutual annihilation. They also required a degree of mutual verification and trust-building that currently has no analogue in AI development, where training runs are invisible, model capabilities are routinely underestimated by their own creators, and distillation allows capabilities to transfer across borders without physical shipments.
The distillation dispute is itself a case study in how fraught verification would be. U.S. agencies this month alleged that Chinese AI companies have systematically copied cutting-edge American models through a process called distillation — essentially training a smaller model to replicate a larger one’s outputs without access to its weights. Beijing denied the accusation. Even setting aside who is right, the episode illustrates that unlike nuclear warheads, AI capabilities cannot be counted, inspected, or reliably attributed. Any future governance regime will have to solve that problem first.
There is a significant irony embedded in this week’s events that neither side’s public statements acknowledged: Beijing’s State Security Minister Chen Yixin, writing in the state-run China Cyberspace magazine on the same weekend as Amodei’s essay, called for China to “accelerate the establishment of a system for preventing and controlling AI security risks” and to “engage extensively in international cooperation.” That framing — domestically focused safety infrastructure combined with a stated desire for international dialogue — is not entirely incompatible with what Theo Bertram, director of the London-based Social Market Foundation, described as “the domestic introduction of some safeguards in both countries.” The gap between what Beijing’s security apparatus privately acknowledges as a risk and what its foreign ministry publicly performs as geopolitical resistance may be narrower than the week’s headline exchanges suggest — and that gap is where any realistic governance conversation would have to begin.
What to Demand — and What the Stakes Actually Are
The Xi-Trump meeting expected on September 24 in Washington will be watched closely as a potential venue for early AI governance signals. Expectations should be modest. Geopolitical summits rarely produce technical agreements, and the distillation dispute has pre-poisoned the well on the question of Chinese compliance with any export-control-adjacent framework.
What a realistic near-term outcome looks like, per Bertram, is domestic regulation in both countries that creates accountability without requiring bilateral trust. The European Union’s AI Act — however imperfect — provides a working model for what mandatory transparency and risk-tiering can look like in practice. Neither the U.S. nor China has moved toward anything comparable, but the political pressure from publics in both countries worried about AI disruption, job displacement, and security risks is real and growing. As we’ve noted previously, governments are beginning to study the economic cost of AI access decisions — evidence that the policy conversation is maturing even if the diplomatic one is stalled.
The harder reality is that competitive pressure — not bad faith — may be the primary driver of the current impasse. Chinese companies now offer AI performance comparable to leading U.S. models at dramatically lower cost, and President Xi has been positioning China as a champion of open-source AI for the Global South. At the BRICS summit in India, Xi offered member nations support in developing open-source AI, framing China’s model as an alternative to the proprietary systems built by most leading American labs. That is a coherent geopolitical strategy, and it makes unilateral restraint by China — absent a compelling mutual framework — genuinely irrational by any standard cost-benefit analysis.
The same logic applies in reverse for the United States. In an environment where the primary political framing is “whoever wins AI wins,” safety advocates are not just fighting technical skepticism — they are fighting a competitive incentive structure that treats any slowdown as a unilateral disarmament.
The Strongest Counterargument
The most credible objection to this week’s risk-framing is that geopolitical theatre is not the same as actual policy, and that beneath the headline exchanges, both governments are in fact building domestic AI safety infrastructure. Critics of the “arms race” narrative — including some AI governance researchers — argue that framing every bilateral friction as an insurmountable barrier overstates the dysfunction and understates the work happening at the technical and regulatory level within each country.
Chen Yixin’s comments about accelerating domestic AI risk controls are a real data point in this argument. So is the fact that Carnegie Endowment researchers and others have documented ongoing technical exchanges between Chinese and Western AI safety communities at the working level, even as the diplomatic temperature rises. The argument is that safety progress does not require a grand bilateral treaty — it can happen incrementally, in parallel, with periodic convergence on standards as each side learns from the other’s domestic experience.
This is a serious point, and it deserves to be taken seriously. Domestic regulation in both countries would be materially better than no regulation in either. But it does not fully address the core risk: that the most consequential AI safety decisions — the pace of capability development, the deployment of autonomous AI agents, the militarization of AI systems — are the ones least likely to be constrained by domestic policy alone, and most likely to require the kind of bilateral trust that this week’s events suggest is not currently available. The counterargument weakens the most alarming version of the risk framing; it does not dissolve the underlying problem.
Tough Questions for the People in Charge
- For Dario Amodei and Anthropic: Given that your essay bundled a safety argument with explicit export-control advocacy targeting China, what specific steps are you taking to ensure your safety proposals can be evaluated separately from their geopolitical framing — and do you believe a version of your proposals could attract Chinese engagement?
- For the Trump administration: If the primary metric for AI policy is competitive dominance over China, what specific governance mechanisms — if any — would you accept as compatible with that goal, and who inside the administration is responsible for answering that question?
- For Beijing’s State Security Ministry: Minister Chen Yixin called publicly for international AI cooperation on the same weekend Beijing’s Foreign Ministry dismissed safety calls as fearmongering. Which position reflects actual Chinese policy — and what would a credible invitation to cooperate on AI safety look like from China’s side?
- For negotiators ahead of the September 24 Xi-Trump summit: Given the distillation dispute and mutual accusations of bad faith, what verification mechanisms, if any, are being discussed — and is any form of binding AI governance agreement realistically on the agenda?
- For the AI safety research community: If bilateral governance is genuinely off the table in the near term, what domestic policy instruments in the U.S. and China would provide the most meaningful safety floor — and are you actively engaging with policymakers in both countries to advance them?
