← Back to blog
NewsAbout 6 min read

METR Spent 11 Weeks Inside Anthropic. The Number Is 1.9, and Most of the Evidence Is Blacked Out.

Published Nov 7, 2026
METR Spent 11 Weeks Inside Anthropic. The Number Is 1.9, and Most of the Evidence Is Blacked Out.

METR published an independent assessment this month of whether AI is accelerating AI research, based on eleven weeks of unusual access inside Anthropic. The headline estimate is that Anthropic's measured capability progress has run at roughly 1.9 times its earlier rate since a change point in October 2025, with a 95 percent confidence interval of 1.5 to 2.8.

In index terms: about 15 AECI points per year before, about 29 after. That is two years of historical progress compressed into one. METR compares the magnitude to the jump from Claude 2.1 to Sonnet 4.5.

The second headline is that roughly 80 percent of the acceleration is attributable to AI being used in Anthropic's own research. The third is that Anthropic redacted the majority of the evidence METR rated most informative.

Those three sentences have to be read together, and almost nobody is reading them together.

What access looked like

From August 12 to October 29, a team of about two and a half full-time staff had access to pretraining and RL metrics, training compute figures, internal benchmark scores from unreleased models, internal acceleration analyses, model forecasts, and around fifteen interviews with Anthropic researchers plus staff surveys. METR took no money from the company and worked on Anthropic devices for security reasons.

The analysis covered three unreleased models that finished training between July and October 2026. That is the part that makes this different from every previous outside evaluation, which could only look at shipped models and public benchmarks.

METR also states a negative finding that matters as much as the positive one: it is confident the acceleration is not explained by faster growth in training compute. Something about how the work gets done changed, beyond how much hardware was pointed at it.

The 80 percent is a judgment, not a measurement

There is no instrument that reads "how much of this speedup came from AI." METR combined benchmark trajectories, training metrics, interviews and surveys, and arrived at an attribution estimate.

The interview material is the most concrete part. One researcher told METR that the work is "now only bottlenecked on ideas and compute, no longer bottlenecked on code." Others reported that experiments which used to take one to two days to set up are now close to free. In informal Slack polls from October 2026, staff estimated the latest internal model made them about twelve times more productive. For comparison, the same measure sat around 1.2 times with Claude 3.7 Sonnet.

METR calls those self-assessments very loose evidence, unrepresentative, and likely to overstate the real gain. Twelve times is the kind of number people report when a task that used to be annoying stopped being annoying, and it does not survive contact with a stopwatch.

So the honest state of the claim is: a large share, likely most, plausibly 80 percent, of a 1.9x acceleration traces to AI assistance. Every word in that sentence is doing work.

The threshold has a number, which is the problem

Anthropic's Responsible Scaling Policy version 3.4 took effect on July 8, 2026. It names two triggers related to automated research: an AI system able to replace the company's research staff at competitive cost, or a doubling in the pace of progress plausibly attributable in substantial part to research or engineering automation. The policy defines a doubling as making in one year the progress that would otherwise take two.

A policy that specifies a factor of two creates an obvious question when an auditor reports 1.97. Anthropic's public August 2026 report said acceleration was "less than a factor of 2." Its own unredacted internal version implies 1.97, according to reporting on the assessment. The company has described crossing the threshold as plausible. In a November 2 filing with the US Securities and Exchange Commission, it said it "may imminently meet" its own definition of dramatically accelerated progress.

METR separately judged it plausible that the early warning threshold had already been crossed by July 15.

Read that sequence again, because it is strange. A company wrote a rule with a number in it. An outside auditor, with internal access, estimated a value just under the number. The company then told regulators it might be about to cross the number. Nobody has said the threshold has been crossed.

That is not necessarily evasion. Anthropic characterizes the threshold as an early warning rather than proof that severe risk has arrived, and it says it is not yet at fully autonomous development of its successors. A policy threshold that fires is supposed to trigger safeguards, and the question of which safeguards is a real one that cannot be answered in a press release.

But it does mean the mechanism currently produces statements rather than actions, and the statements are carefully hedged.

The redactions are the story

METR says Anthropic withheld the majority of the material the evaluators found most informative, including future model forecasts and bottleneck analysis. A planned portion of the audit covering how the company divides compute between capability and safety research was cancelled midway.

The consequence is specific. The public cannot check the estimate. The 1.9 figure depends on benchmark selection, on where the change point is placed, and on assumptions about task difficulty. METR's own sensitivity analysis moves the estimate between 1.1 and 2.9 times depending on those assumptions.

And there is a second number that gets less attention. When METR compared Anthropic against frontier models from all developers rather than against Anthropic's own history, the estimated pace rose from about 17 to 24 points per year, or 1.4 times. METR suggests the difference could reflect Anthropic catching up from behind the frontier, or differences in how earlier models were tuned to benchmarks.

1.9 for one company. 1.4 for the field. Both are accelerations. Only one of them is close to triggering a policy written around a factor of two.

What would make this checkable

The honest answer is that nothing in the published material lets an outsider reproduce the analysis, and that is a structural feature of evaluating a private company rather than a flaw in METR's work.

What could change it: publishing the AECI benchmark composition and weights, so the index means something outside the building. Releasing the change-point methodology. Letting a second evaluator see the redacted material under the same terms. And separating the two questions that are currently fused, which are whether progress accelerated and whether AI caused it. The first is measurable. The second is inferential.

There is also a question METR raises and does not answer. It found no bottlenecks that would slow the pace over the next twelve months, which means further acceleration is a live risk rather than a settled question. If the current rate holds and compounding continues, the factor of two stops being a threshold and becomes a description.

For anyone outside the labs, the practical takeaway is narrower than the headlines suggest. AI is currently an accelerant for human researchers. It is not doing autonomous research. The gap between those two statements is where all the policy argument actually lives, and METR's report, redactions included, is the most precise description of that gap anyone has published.

Related articles