METR Report: Anthropic's Model Capability Progress Is About 1.9 Times Faster Than Before, With 80% of the Increment Coming From AI

Add crowdsourcing promotion copy Add crowdsourcing promotion copy end Add App-exclusive traffic redirect

Editor｜Zenan

On November 5, the third-party evaluation organization METR released an investigation report on Anthropic stating that the pace of benchmark capability progress this company achieved over the past year roughly equals the progress of the previous two years.

From August 12 to October 29, this organization spent about 11 weeks going inside Anthropic to review its model results, training compute, R&D efficiency and other materials, and conducted interviews with researchers.

Their analysis used Anthropic's version of the comprehensive capability metric AECI. Data shows that since October 2025, the rate of progress of large models on this metric is about 1.9 times the previous rate, with a 95% confidence interval of 1.5–2.8 times.

Since October 7, 2025, this score has jumped from about 15 points/year to about 29 points/year, covering in one year the ground previously covered in about two years, roughly equivalent in magnitude to the span from Claude 2.1 to Sonnet 4.5. Anthropic's own risk report had previously assumed that Mythos Preview was a single jump, after which the trend would return to the original one; however, now new models have appeared, and the new slope continues.

Looking across the whole industry, the acceleration of frontier models is only 1.4 times.

In the interviews, a recurring phenomenon was that researchers can finally try more ideas.

In the past, an experiment might take one or two days to implement, and time and compute costs were insurmountable, so many ideas were shelved before they could be validated. Today, researchers can hand multiple ideas to Agents and see which ones are worth pursuing further.

One researcher described that their work is now mainly limited by ideas and compute, and code is no longer the bottleneck. Others feel that after using Claude, it is as if they have a postdoc and a graduate student advancing projects together with them. Experiments and implementation have become cheap, making researchers more willing to try, and many small experiments previously deemed not worth the effort now have a chance to become new research directions.

METR's report also revealed some internal progress previously invisible to the outside world. Their analysis included three Anthropic models that have not yet been made public, referred to as Model 2, Model 3, and Model 4. They completed training between July and October this year.

The strongest currently is this Model 4 (some guess it is Claude 6), whose scores on many common benchmarks already exceed 90%. In October, Anthropic employees made internal subjective estimates of it, voting that on average it improved their work efficiency by about 12 times.

These models have not been released, and may not even become public products, but their existence reminds the outside world that looking only at launch events and public leaderboards may not fully capture what is happening inside major AI companies. From about 1 time in early 2025, to about 2.5 times for Opus 4.5, about 5 times for Mythos 5, and a jump to 12 times for Model 4, the exponential fit is 3.9 times per year.

METR estimates that as of October this year, about 80% of the current acceleration in AI research is estimated to come from AI. There is another 20% from non-AI factors, and Anthropic did not allow the details to be disclosed.

At present, there is still a gap before achieving complete recursive self-improvement (RSI) in which AI independently develops the next generation of models. Anthropic's own researchers believe that in the most important part of model training, AI has not yet made highly important and fully autonomous contributions, and the main gains currently still come from AI-assisted human R&D.

However, Anthropic researchers have already described a large number of autonomous contributions from AI as "signs of life," and there is quantitative analysis and statistics on the proportion of tasks completed by AI versus humans in each stage, but the data was deleted by Anthropic.

As long as stronger models can help researchers develop the next generation faster, the feedback process may continue to strengthen. How far the RSI path can go depends on the links that have not yet been automated, and on whether resources such as compute can keep up.

Regarding predictions for the near future, METR does not see any bottleneck in the next 12 months that would clearly block rapid progress, but also admits that as information increased, they changed their judgment multiple times, and their confidence is currently limited.

In addition, although Anthropic allowed the third-party organization METR to enter for an internal investigation, the 100-page report currently provided has undergone substantial deletions by Anthropic. Most of the most persuasive evidence was deleted by Anthropic itself.

A companion review released the same day also showed differences between the two sides in risk wording: Anthropic once described a certain leading indicator as showing a rate of progress "less than twice," while METR calculated about 1.97 times based on its internal data and believed the more accurate description should be "about twice." The twofold acceleration involves Anthropic's early warning threshold.

METR believes that the possibility cannot be ruled out that the relevant threshold had already been crossed as of July 15, but most relevant researchers still lean toward it not having been crossed at that time. Both sides also agree that the direct catastrophic risk of the relevant models at that time was low.

Reference content:

]article_adlistarticle_adlist[

For reprinting, please contact this official account for authorization

Submissions or reporting requests: liyazhou@jiqizhixin.com ]article_adlist Statement: This is an exclusive article for Sina.com; unauthorized reproduction is prohibited. tech_web_article_v2015_block_bottom non-targeted 300*250 button 17/09 wenjing begin

non-targeted 300*250 button end