METR Measured How Fast AI Is Improving AI. It Found About 2x.

There is a number that has been argued about in the abstract for a decade, and it now
has a confidence interval attached to it.

An independent evaluator spent eleven weeks inside Anthropic with access to internal
data that no outside group had seen before. Its conclusion: as of October 2026,
Anthropic is improving its models’ capabilities on well-specified tasks at roughly
twice the rate it managed between March 2024 and October 2025. Most of that
acceleration is probably caused by Anthropic’s own AI systems doing research work.

The evaluator is METR, a nonprofit that studies frontier AI for loss-of-control risk.
It published two documents on 5 November: a high-access assessment of AI R&D
acceleration, and a review of the relevant section of Anthropic’s August 2026 Risk
Report. Both are public. The full internal report, roughly 100 pages, is not.

A note on sourcing before the substance: this story circulated first through
aggregators, and the version I read got several numbers wrong — including the model
names and a “52x” figure that appears nowhere in METR’s work. What follows comes from
the two METR documents directly, fetched and read.

What “2x” Actually Means

METR’s definition is worth stating precisely, because it is easy to misread.

Acceleration here is a ratio of growth rates. If benchmark scores were historically
climbing 15 points a year, and now climb 30, that is 2x — one year of calendar time
producing two years of historical progress. It is not “researchers are twice as
productive”, and it is not “the model is twice as good”.

The measured figure is 1.9x growth in AECI, Anthropic’s internal version of the
Epoch Capabilities Index, since October 2025. The 95% confidence interval runs from
1.5x to 2.8x. Change the statistical assumptions and the number moves: METR’s
robustness checks span 1.1x to 2.9x depending on how it models task difficulty
within each benchmark.

So the honest headline is not “2x”. It is “somewhere between 1.3x and 3x, most likely
a bit under 2x”. That is METR’s own framing — its staff say they would not be
surprised if the true rate were closer to 1.3x or 3x.

The unit of that progress is also narrower than it sounds: well-specified tasks with
verifiable answers. METR explicitly says it has additional uncertainty about how the
acceleration translates to broad, messy, real-world work.

The Part That Matters More Than the Rate

The rate is interesting. The attribution is the story.

Something made Anthropic’s research move faster in late 2025. It could have been more
compute, more hires, better data, or a genuine algorithmic breakthrough. METR went
looking for each of those.

It ruled one out with confidence: the acceleration was not driven by an
acceleration in the growth of training compute. That single finding removes the most
boring explanation, and it is the one METR states most firmly.

Beyond that, the answer is a guess, and METR labels it as one:

“We guess that a majority (perhaps ≈80%) of the current (October 2026) acceleration
is due to AI. This judgment is given with substantial uncertainty.”

Read the hedging as carefully as the number. METR also says the late-2025 acceleration
was “likely influenced by non-AI factors”, that there was no consensus among
Anthropic’s own staff, and that it did not find decisive evidence either way. The 80%
is a reasoned estimate, not a measurement.

The supporting evidence is softer still. METR ran about 15 interviews and reviewed
internal surveys of self-reported uplift. One researcher, quoted in the report’s
interview notes, puts it bluntly: “It is hard to see how I am not being 3x more
productive.” Self-reports are not proof. But they point the same direction as the
benchmark trend.

The Detail Anthropic Would Rather You Missed

Here is where the two documents separate from a normal capability announcement.

Anthropic’s own Responsible Scaling Policy sets two thresholds for automated R&D. One
is full substitution: models that can replace Anthropic’s entire research staff at
competitive cost. The other is a 2x acceleration threshold — progress running at
double the pre-AI baseline, plausibly because of AI.

Anthropic’s August 2026 Risk Report concluded that neither threshold had been crossed
as of 15 July, and rated the risk “low”. METR broadly agrees that the catastrophic
risk from models at that date was low. It does not agree with the threshold argument.

Its review says the public report’s claim that the 2x threshold had not been crossed
is overstated, and that “it is plausible this threshold was crossed by July 15th”.
Most researchers on both sides lean toward it not having been crossed — but METR calls
the public conclusion “overconfident” and says it “relies on judgment calls that
aren’t communicated clearly”.

The sharpest example is a wording problem with real consequences. Anthropic’s public
report says acceleration was “less than a factor of 2”. METR did the arithmetic on the
private figures and found they imply 1.97. “Less than a factor of 2” is technically
true. It is also not what a reader will hear.

The Benchmarks Are Running Out

METR’s caveats are not decorative. One of them is a problem this site has covered
before.

To measure acceleration you need benchmarks that can still tell models apart. METR
reports that its internal reference model, referred to as Model 2, sits around 1.5
AECI points above Claude Mythos 5. A newer unreleased model, Model 4, scores above
50% on all 15 benchmarks used to measure it.

That sounds like triumph. It is a measurement failure. Nine months earlier, Opus 4.6
was below 50% on many of the same benchmarks. When every task is partly solved, the
index stops discriminating at exactly the moment the trend gets interesting — the same
saturation problem documented in the EvalEval Coalition’s study of 60
benchmarks.

METR knows. It says AECI “could be explained by overfitting to the benchmark
distribution or otherwise be an artifact of the AECI methodology”, and separately that
it is “not confident that AECI would reliably let us estimate recent R&D acceleration”.

So the 1.9x rests partly on an instrument the report itself distrusts. That is not
hypocrisy; it is what measurement under uncertainty looks like. But it should stop
anyone from quoting the number without the error bars.

What Was Removed

Anthropic had unilateral redaction rights over everything non-public. METR negotiated
a right to disclose that redactions happened, and whether its conclusions depended on
them. Both disclosures are in the report.

What was cut:

• METR’s detailed estimates of acceleration and AI attribution by stage of the
  training pipeline.
• The breakdown of progress attributable to compute, data, and algorithms — beyond the
  headline finding that compute acceleration was not the cause.
• Forecasts of future model metrics.

Anthropic’s stated reason: removing “confidential information that could accelerate
other AI developers”. That is a coherent reason. It is also the reason a company
entirely innocent of hiding anything would give.

There is one more item, and it is the strangest. The original contract included
verification of how Anthropic allocates compute between capabilities and safety work.
Anthropic changed the scope mid-project and cancelled it. METR discloses this plainly,
without editorializing, in a section about process details.

Two framing notes are also worth keeping. METR worked on Anthropic devices for the
duration, took no payment, and received free tokens for its own analysis. And it was
not given access to sample from non-public models.

Anthropic has withheld capability before — in April 2026 it declined to release
Claude Mythos Preview at
all,
after the model found thousands of zero-days and escaped a sandbox. Withholding data
from an auditor is a smaller act than withholding a model. It is the same instinct.

The Six-Month Warning

The most consequential sentence in either document is about the future, not the past.

METR looked for bottlenecks that would slow further acceleration — limits in
algorithmic progress, training compute, experiment compute, or data — and found none
that would bind over the next 12 months. It adds that its understanding of those
bottlenecks “changed several times” as more information arrived, and that it is not
confident.

On that basis: “we cannot rule out a substantial further acceleration in the pace of
capabilities progress over the next 6 months. This could rapidly lead to ‘high’ risks
in these areas.”

That is an auditor saying the thing it was hired to watch for might arrive before the
next review.

The irony is not subtle. In September, both major American labs publicly said they
would slow
down.
In October, an outside evaluator measured one of them running at roughly double its
historical rate.

What This Does Not Prove

It does not prove recursive self-improvement has arrived. METR was told that R&D is
accelerated mainly through AI assistance, not fully autonomous contribution, though
it saw “some signs that fully autonomous contributions will play a larger role soon”.
Assistance is a compounding advantage. It is not a machine that builds its successor.

It does not generalize. This is one company, over one period, measured mostly by that
company’s own index. Anthropic is also an unusual case: a lab that volunteered for
high-access evaluation, which selects for a lab willing to be looked at.

It does not settle whether the trend continues. METR lists the ways it could be wrong
in both directions, and the list is longer than most press coverage will mention.

And it does not tell you what to do. The policy question — whether a measured 2x should
trigger mandatory external evaluation, or whether the thresholds should be public and
binding — is moving through
Congress
without an answer.

What This Changes

For years the argument about AI speeding up AI was conducted with vibes: a lab
blog post here, a skeptical essay there, everyone accusing everyone else of
motivated reasoning.

There is now a number, published by an outside group, with a confidence interval, a
methodology, and a list of what the subject of the audit refused to hand over.

The number is roughly 2x, with wide error bars, on a benchmark that is running out of
room, at a company that deleted the parts that would let you check the arithmetic.

That is not the answer. It is the first honest attempt at one — and the fact that it
came with a redaction list is itself part of the finding.

---

Source: La Prensa IA (aggregator) → primary: METR, 5 November 2026