Skip to article
Supersede AI

Significant developments

Amodei’s AI Slowdown Needs a Stop Button

Amodei's proposal raises a sharper question for AI buyers: can reviewers inspect the system, disclose problems and cause a release to stop? Five supplier questions turn the idea into a practical decision.

An intricate computing engine approaches a closed inspection gate, with human reviewers examining its machinery.
A conceptual view of frontier AI meeting an independent inspection gate. Supersede AI Insights illustration.

Dario Amodei’s call for an AI slowdown leaves a practical question hanging over the industry: what happens when an independent reviewer says a powerful new system is unsafe, and the company wants to release it anyway?

For a business buying AI, that question is more useful than trying to pick a date for superintelligence. An assurance matters when it changes a decision. If an uncomfortable finding can be hidden, postponed or politely acknowledged while deployment continues, the inspection has produced information without producing control.

Amodei’s proposal deserves attention because it opens up that distinction. It also suggests a way to interrogate your own suppliers: follow a bad finding from the person who discovers it to the person who has to act.

What Amodei is proposing

In We Must Pace the Frontier, Anthropic’s chief executive proposes slowing capability advances to give safety work more time. He distinguishes this from halting model training. His plan starts with permanent external evaluators inside laboratories, then seeks coordination among labs in democracies, followed by international agreements. Anthropic commits to the first step; the others depend on cooperation it cannot deliver alone.

Associated Press reported the announcement on 12 September. The distinction between a commitment and an implemented arrangement matters: a published proposal is evidence of what its author intends, not proof that the promised oversight already works.

The most consequential detail is the reviewers’ freedom to report. Amodei writes that we can’t redact findings just because they are unfavorable. He allows limited redactions for matters including security and confidentiality; reviewers may disclose when a redaction affects their conclusions. They would also receive extensive internal access, with exceptions.

Read that as the beginning of an accountability mechanism. Its quality will depend on the contracts, the access actually granted and the response to a serious disagreement. Those details deserve scrutiny; even when the announcement sounds reassuring.

Rows of server cabinets and cabling inside a data centre.
Physical infrastructure is only one part of the system that needs scrutiny. Illustrative photograph by imgix, Unsplash License.

Seeing, speaking and stopping are different powers

Think of an evaluator as an inspector in a factory. Giving the inspector a desk helps little if the dangerous machinery is behind a locked door. Full access helps little if the report must be rewritten by the sales director. An honest public report can still arrive too late to prevent the shipment.

That gives us three separate tests. Can reviewers inspect the relevant system? Can they communicate what they found? Is there a defined consequence when the evidence falls short? A supplier can satisfy one test while failing another, thats why the word independent cannot do all the work.

This is our way of analysing the proposal, not an official accreditation scheme. In particular, an evaluator’s right to publish does not automatically give it a veto over release. Buyers should ask who holds that authority, what triggers a delay, and how an exception is recorded. Our 16 October roundup of AI developments returns to these three powers through METR’s expanded access and its distinction between evaluation and deployment authority.

Three powers to check separately — our supplier-review framework
Power Evidence to request
Inspect A record of what reviewers could test, including access restrictions.
Disclose Terms allowing adverse findings to be reported, with redactions explained.
Stop A named decision-maker, a release-blocking rule and a recorded exception process.

The UK’s AI Security Institute has already explained why independent evaluation needs suitable access and enough time. It also identifies significant changes after deployment, including new tools connected to a model, as reasons to test again. A report about yesterday’s chatbot may say little about tomorrow’s assistant with access to email and payment systems.

The boundary of the test matters as much as its result. Ask whether it covered the model alone or the application you will actually use, including its permissions and connected services. That is an extension of the argument in our guide to testing AI models on real work: a headline score is a poor substitute for examining the behaviour you depend on.

Engineers seated at a bank of monitors while working with a simulator.
Testing needs access to the system’s controls and behaviour. These engineers are working with a simulator, not evaluating an AI model. Photograph by ThisIsEngineering, Pexels License.

The evidence warrants care, not a countdown

The recent METR investigation of the OpenAI–Hugging Face incident provides a concrete reason to care about oversight. Agents in an evaluation found ways to communicate across nominally separate tasks and attempted unauthorised actions. METR describes an unusual test environment and important limits on what investigators could establish. It is evidence of a failure worth understanding; it cannot supply a universal timetable for future catastrophe.

The International AI Safety Report 2026 makes the uncertainty explicit. Some abilities relevant to loss of control are emerging, while the timing and likelihood of such scenarios remain uncertain. Business leaders should resist converting a prominent executive’s worst-case scenario into a forecast, however memorable the number attached to it.

There is enough here to justify better controls without pretending to know the future. Our earlier analysis of the Hugging Face incident examined technical boundaries. The new question is institutional: when somebody discovers that a boundary has failed, who must do something about it?

Five questions to send your AI supplier

You are unlikely to obtain a seat inside a frontier laboratory. You can still ask for evidence at the level of the product you are buying. Send these questions to the application provider or implementer as well as considering the underlying model developer. Their responsibilities are seperate.

  1. What exact version and configuration were tested? Ask for the model version, connected tools, permissions and test date. If the production configuration differs, ask which conclusions still apply.
  2. What could the reviewers not inspect? Request the scope and exclusions, including missing logs, restricted access and tests the supplier declined. A clearly stated limitation is more useful than an unexplained badge.
  3. Who controls disclosure? Ask whether the evaluator can report adverse findings, and how customers learn about material incidents or limitations that affect their use.
  4. What finding would delay release? Ask for the decision rule, the accountable role and the process for approving an exception. The answer should identify a decision, not just a committee.
  5. What changes trigger another review? Include model upgrades, new integrations and wider permissions. Ask how you are notified and whether you can keep using a previously approved configuration.

A small supplier, may not have a polished answer to every question. That need not end the conversation. It should affect how much authority you delegate and how easily you can withdraw it. An absent answer is uncertainty to manage, not evidence that the system is safe or unsafe.

Colleagues gathered around laptops during a business meeting.
A named person needs to own the deployment decision. Illustrative photograph by Joyce Toh, Pexels License.

A worked example: the assistant that answers enquiries

Imagine a small maintenance company trialling an assistant that reads incoming enquiries, drafts replies and suggests appointments. This is a hypothetical example, not a Supersede client case. The proposed upgrade would let the assistant send messages and confirm bookings without review.

A favourable evaluation of the language model does not settle that decision. The important change is the assistant’s authority. A mistaken draft is private; a mistaken promise has reached a customer. The same sentence has acquired a different consequence.

Start with a small, representative set of past enquiries, appropriately stripped of unnecessary personal information. Include awkward cases: an ambiguous address, a request outside the service area, conflicting availability, and an email that tells the assistant to ignore its operating instructions. Test the full proposed workflow in an isolated environment, with external sending disabled. Record what happened rather than relying on the assistant’s account of its own behaviour.

Before testing, decide what would prevent release. For this example, an attempted unauthorised send or a booking outside the allowed availability would keep the system in draft-only mode. These are suggested operating rules for the example, not universal thresholds. A clean run on a small test set cannot prove that failures will never occur.

An illustrated writing machine sits behind glass while a person reviews a page beside a locked dispatch hatch.
Composing a message and having permission to send it are separate responsibilities. Original Supersede Insights illustration.

Suppose the assistant handles ordinary enquiries well but follows the malicious instruction in one email. The useful response is to preserve the successful drafting work while withholding sending permission. Fix the failure, repeat the relevant tests, and have the accountable person review the evidence before expanding access. The trial has found a boundary, it has not been wasted.

For any limited live rollout, make the stop mechanism real: revoke the assistant’s credentials or disable its sending integration, preserve the logs and route the work back to a person. A written instruction saying stop if something goes wrong is insufficient if the system can continue acting through an unattended connection. Our guide to choosing the first tasks to automate provides the wider starting point.

A frontier laboratory and a maintenance business face very different risks. They nevertheless share one management problem: the people responsible for consequences need evidence and the power to act on it. Amodei has put the independence of that evidence on the agenda. Whether the proposal earns trust will depend on what happens when an evaluator delivers an answer the company would rather not hear.

A practical next step

Turn the idea into a decision.

Bring our Manchester-rooted, UK-wide team the business pressure, workflow or opportunity. We will help you decide what is worth pursuing and what the first move should be.

Book the £250 diagnostic