AI Capability Update: Three Trend Lines, No Single Rate

Abhishek Dash12 min read
Three divergent lines, graphite, red pencil and blue thread, run across a sheet of cream paper beneath a tarnished brass caliper, a folding wooden rule and a coiled paper measuring tape.

In short

METR published its Q4 2026 capability update on 20 November 2026. On its combined TH1.1 and TH2 suite, the 50% time horizon doubles every 3.61 months (95% CI 2.08 to 4.94). The same report finds no strong acceleration on an index needing no human time estimates. OpenAI reports 16.4 OECI points per year and Epoch 14 ECI points per year. Different tasks and units, so they cannot be merged.

Key takeaways

  • METR published its Q4 2026 capability update on 20 November 2026, its first new task suite since Time Horizon 1.1 on 29 January 2026.
  • On the combined TH1.1 and TH2 suite, the 50% time horizon doubles every 3.61 months (95% CI 2.08 to 4.94), against 105 days for TH1.1 alone.
  • METR's IRT-based capability index shows no strong acceleration or deceleration, which the report grades as weak to moderate evidence against a general acceleration.
  • On large human pools, the strongest models reach the 96th to 98th percentile but do not top the leaderboard; on Budget NanoGPT the best agent run reached a 15.7% speedup in 48 hours.
  • No model-designed evaluation was judged useful by METR staff, and on the philosophical-argument task models scored below humans against a single rater.
  • OpenAI reported 16.4 OECI points per year on 18 November 2026 and Epoch 14 ECI points per year on 1 September 2026; a point is not a month and no stable conversion is published.

On 20 November 2026, METR reported that the length of task an AI agent can complete with 50% reliability doubles every 3.61 months, with a 95% confidence interval of 2.08 to 4.94 months. The figure came in the Q4 2026 capability update, the organisation's first new task suite since Time Horizon 1.1 in January 2026. In the same report, METR said its index that does not rely on human time estimates showed no strong acceleration and no strong deceleration.

Both statements sit in the same document. Nearly all the attention went to the first. The second, and the judgment-task results beside it, tell a reader what the number can and cannot support.

What METR added in November

Time Horizon 1.1, published on 29 January 2026, measured the duration of software tasks that models and humans could complete at 50% success. It covered 170 to 228 tasks, weighted longer tasks more heavily, and put the doubling time since 2023 at about 131 days. The Q4 update keeps that suite and adds a second, called TH2, with two changes that matter here.

One, TH2 includes tasks that are expensive or awkward to turn into a containerised reinforcement-learning environment, plus tasks where a good answer needs judgment rather than a fast empirical check. METR calls the first group hard-to-verify and the second judgment-laden. The split is the reason the new suite exists.

Two, METR fitted an item response theory model across public frontier models to produce a capability index it calls MCI. An IRT index places models and tasks on a common scale without asking people how long the tasks take. That makes it a check on the time-horizon metric, which does rest on human estimates for 91% of its tasks.

On verifiable tasks, the models won

On coding and optimisation tasks with a clear check, frontier models matched or beat the skilled people METR recruited, and METR usually could not find humans who beat the models. The caveat on that sentence matters. METR spent roughly 90,000 dollars on seven attempts to recruit human baseliners, paid between 200 and 350 dollars an hour, and said in the report that the baseliners' skill probably did not match a typical frontier AI researcher. Direct human baselines feed only 9% of METR's time estimates.

Where a large pool of human participants already existed, the result is sharper. In Al Zimmermann contests and similar open competitions, the strongest models, named in the report as Fable 5.1, Opus 5.5 and GPT-6 Astra, placed at the 96th to 98th percentile. In the median contest, roughly four humans still beat them. On Budget NanoGPT, the best agent run in 48 hours reached a 15.7% speedup, while the human record at position 92 reached 41% with AI assistance over an estimated 60 hours.

Those are two different comparisons, and both are checkable. Against the people METR hired, models win. Against an open field, they place near the top and do not take first place.

The doubling time barely moved

On the combined TH1.1 and TH2 suite, the 3.61-month doubling works out to about 110 days, against 105 days for TH1.1 alone. On TH2 verifiable tasks by themselves, the doubling drops to 1.65 months with a confidence interval of 0.79 to 3.99, which overlaps the combined figure. Fit the same verifiable trend back to GPT-4o and it comes out near 3.2 months, not 1.65.

METR treats the shorter figures with care. It notes that MirrorCode showed a similarly fast doubling in late 2025 while TH1.1 was slower, and calls it plausible that shorter doubling times on less diverse task sets reflect task similarity rather than general acceleration.

The confidence intervals deserve as much space as the point estimate. The combined doubling's interval, 2.08 to 4.94 months, spans a factor of 2.4. METR also states that measurements above 16 hours are unreliable with its current tasks. The report's naive extrapolation reaches a 200-hour time horizon in July 2027, and METR labels it naive precisely because it assumes the trend continues past the point where the instrument is sound. July 2027 is an extrapolation rather than a finding the report supports.

The result that does not need human hours

The capability index is the part of the update hardest to fold into a headline, and the most useful. Across public frontier models, METR's MCI shows no obvious acceleration or deceleration. The report grades this as moderate evidence against a general deceleration and weak to moderate evidence against a general acceleration. Per-developer fits do show slope changes for Anthropic models, but METR says these are uncertain and quite sensitive to methodological choices.

The time-horizon metric asks people to estimate how long tasks take, and METR has warned that cheap containerised evaluations resemble the reinforcement-learning environments used to train the models being measured. In section 1 of the update, it says that makes eval scores liable to overestimate capabilities in the areas of interest. The MCI does not lean on human time estimates, which makes it a different kind of check on the same models.

METR tried to price the problem with a Lifecycle Cost proxy: the cost of building a task container and running and scoring it 100 times, estimated by an Opus 5.5 judge calibrated against METR's own figures on about ten tasks. Cost and human completion time correlate at R squared 0.82. Holding length fixed, a doubling of lifecycle cost carries a difficulty penalty of about 4.6 months of progress, roughly half the agent's time horizon. The high-cost subset is 2.9 times as expensive, and models perform 2.9 times worse on it, equal to 1.5 doublings.

METR reports that high-cost and low-cost tasks are improving at similar rates. If that holds, the contamination penalty moves the absolute level of capability without steepening the slope. The exponential survives the correction. The claim that an agent can work for a particular number of hours does not, especially beyond 16 hours.

Judgment is the gap, with a large asterisk

On tasks where a good answer needs judgment and there is no fast empirical feedback, models fell below human experts. In the Making METR Tasks exercise, not one evaluation designed by a model was judged useful by METR employees. Opus 5.5 came in borderline above the bar, which METR describes as not usable for its work. On LMCA, a task that scores critiques of philosophical arguments against expert ratings, models performed below people.

Read that as METR measured it, then read the caveats, which METR prints itself. TH2 contains only four hard-to-verify or judgment task families, several with one or two tasks. On LMCA, one person rated most of the arguments, and agents were scored by agreement with that individual. METR's own staff disagree about whether scoring philosophical arguments is relevant to automating AI R&D. The eval-design task has a referee problem: the incumbent evaluator decides whether a model's proposed evaluation is useful, so a genuinely different proposal can score as useless by construction. METR also finds that model judges overestimate the quality of work produced by similarly strong or stronger models.

That does not erase the deficit. Sayash Kapoor of UC Berkeley argued, as MIT Technology Review reported on 20 November 2026, that research taste could be a major bottleneck on the path to automated AI R&D. Others, including Dario Amodei, have argued that models trained on verifiable tasks will eventually handle open-ended ones. The deficit is four task families and one rater away from a durable capability ceiling, and this post reports it at that strength.

The OpenAI report two days earlier

On 18 November 2026, two days before METR's update, OpenAI published Research acceleration: The jagged frontier. It reported that agent success without human intervention on internal research tasks lasting two to four hours rose from 23% in March 2026 to 46% in October 2026. On tasks lasting four to eight hours, the rise was from 13% to 35%. The company's median researcher was spending 2,400 dollars a day on agents by October, up 62% from September, with the 90th percentile at 23,000 dollars a day.

OpenAI also published the unevenness. Success rates ran from 79% on monitoring training runs to 35% on launching training runs, 25% on writing inference and serving code, and 29% on building evaluation, RL and agent environments. The company called the uplift jagged. In interviews, researchers said that for GPT-6 Astra and GPT-6.2 Astra, people came up with the important ideas and models helped implement them. OpenAI stated directly that agents have poor taste in research ideas, code quality, data quality and writing.

This is the strongest disconfirming data for a clean verifiable-good, judgment-bad split. Building an evaluation environment is a verifiable engineering task, and it sits among the two worst categories in OpenAI's own table. The pattern looks more like short and well-specified versus long and integration-heavy, which is a different claim with different consequences for anyone planning to automate research.

OpenAI put its own trend at 16.4 OECI points per year since August 2025 and noted that this sits close to Epoch's 14. METR, reading the same internal task data, derived a 3.8-month doubling and called it similar to the 3.6 it found on its own suite. Two groups looking at the same internal tasks arrived at 3.8 months and 16.4 points. They used different instruments, so the two figures do not compete.

Three trend lines, three units

Three serious groups now publish capability trends, and the numbers cannot be stacked. METR reports a 3.61-month doubling on its own tasks and no acceleration on MCI. OpenAI, on 18 November 2026, reports its OECI frontier rising 16.4 points per year since August 2025. Epoch, on 1 September 2026, reports its ECI frontier rising 14 points per year since reasoning models arrived in September 2024, against 6 points per year for non-reasoning models.

Group and date Figure Window What the unit measures
METR, 20 November 2026 3.61-month doubling, 95% CI 2.08 to 4.94 Combined TH1.1 and TH2 suite Calendar time per doubling of a human-estimated task horizon
METR, 20 November 2026 No strong acceleration Public frontier models, MCI Slope of an IRT index fitted without human time estimates
OpenAI, 18 November 2026 16.4 points per year Since August 2025, internal OECI Points on an index of OpenAI research tasks
Epoch AI, 1 September 2026 14 points per year Since September 2024, public ECI Points on an index spanning many public benchmarks

A point on ECI is not a month of time horizon. At ECI's launch, Epoch said a five-point gain appeared to roughly correspond to a doubling of the METR time horizon, and it now cautions that the mapping is not a stable conversion. Comparing 16.4 with 14 is a comparison of two index slopes on overlapping but different benchmark mixtures. OpenAI notes its models score higher on OECI than on ECI, likely because of the benchmark mixture.

None of that resolves which trend is right, because they answer different questions. A reader quoting one of them as the rate of AI progress is choosing a unit, not reporting a fact. This post does not rank them, and it does not treat the close fit between 16.4 and 14 as proof that the underlying measurements agree.

Where this could be wrong

The judgment deficit is reported as METR measured it, not as a capability ceiling. Four task families, several with one or two tasks, one rater for most LMCA ratings, and a design where the incumbent evaluator decides what counts as a useful evaluation. More raters might change the model ranking. The research behind this post could not check that.

METR's null MCI result is graded weak to moderate evidence against acceleration, not proof of no acceleration. The report lists two ways the method could miss a real one: developers delaying public releases, and the index carrying the most uncertainty about the most capable model.

The inference that lifecycle-cost contamination shifts level rather than slope is the weakest link in the argument above. METR's data supports similar rates of progress on high and low cost subsets, but its task range is narrow and the report says so.

The human comparison is weaker than it sounds. METR says its baseliners probably were not representative of frontier lab researchers, and 91% of its human time estimates are estimates rather than direct measurements. The phrase models beat the humans we hired should not be read as models beat frontier researchers.

Nothing here can adjudicate the three trend lines. There is no published conversion from ECI points to METR months, and Epoch warns that its slope values are only meaningful relative to each other. Any cross-metric comparison in this post is ordinal.

METR's independence is disclosed and constrained, not settled. Its own Hugging Face investigation reports that OpenAI set the scope window, held redaction rights, and gave feedback on structure, emphasis, clarity and tone. Its conflict of interest policy took effect for projects beginning on or after 9 October 2026. METR says it has not accepted funding from frontier AI companies, and a public-records reconstruction found no such transaction in checked filings. METR also says it raised about 71 million dollars in commitments in six months, while only one compatible public grant, 350,000 dollars from the Packard Foundation in July 2026, could be identified. That gap is not evidence of wrongdoing. It is evidence that the funding cannot be traced from outside, and this post does not try to close it.

Finally, this post converts a finished research dossier without fresh reporting. It is shorter than the house length and says so rather than padding. Every figure above traces to the sources listed here, and the ones METR prints with an interval are printed with the interval here.

On this page

Sources

  1. AI R&D-relevant model capabilities are improving rapidly and are getting harder to measureMETR, 2026
  2. Research acceleration: The jagged frontierOpenAI, 2026
  3. The ECI frontier has advanced by 14 points per year since the introduction of reasoning modelsEpoch AI, 2026
  4. Epoch Capabilities Index (methodology and documentation)Epoch AI, 2026
  5. The most misunderstood graph in AI just got a major updateGrace Huckins, MIT Technology Review, 2026
  6. Time Horizon 1.1METR, 2026
  7. Metrics of Agent AbilityMETR, 2026
  8. Our approach to working with AI companiesMETR, 2026
  9. Conflict of interest policy (version 1.1)METR, 2026
  10. Funding updateMETR, 2026
  11. Brief independent investigation of agents' behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incidentMETR / Redwood Research, 2026

Frequently asked questions

What did METR's November 2026 capability update add?

METR added a second task suite called TH2 on 20 November 2026, its first new suite since Time Horizon 1.1 in January 2026. TH2 includes tasks that are expensive to turn into reinforcement-learning environments and tasks where a good answer needs judgment without fast empirical feedback. METR also fitted an item response theory model across public frontier models to build a capability index called MCI, which does not use human time estimates.

Is AI capability doubling every 3.6 months?

On METR's combined TH1.1 and TH2 suite, the 50% time horizon doubles every 3.61 months, with a 95% confidence interval of 2.08 to 4.94 months. That is a result for one task distribution, not a universal rate. METR says measurements above 16 hours are unreliable with its current tasks, and it labels the extrapolation to a 200-hour horizon in July 2027 as naive. The same report finds no strong acceleration on its MCI index.

Why do METR, OpenAI and Epoch report different progress rates?

They measure different things. METR publishes a doubling time in calendar months for a human-estimated task horizon, plus a separate index slope. OpenAI reported 16.4 OECI points per year on 18 November 2026 and Epoch reported 14 ECI points per year on 1 September 2026, both slopes on different benchmark mixtures. Epoch publishes no stable conversion from points to METR months, so the three figures cannot be merged.

Are frontier models worse than humans at judgment tasks?

On the four hard-to-verify and judgment task families in METR's TH2 suite, models scored below human experts. No model-designed evaluation was judged useful by METR staff, and on the philosophical-argument task models fell below people graded against one rater. METR and outside researchers caution that four families, one or two tasks in some of them, and one rater are thin ground for a durable capability ceiling.

What is a 50% time horizon in METR research?

It is the duration of a task that a model completes successfully half the time. METR estimates it from tasks that people could do within a given span, and only 9% of those human-time estimates come from direct human baselines. METR states that horizons above 16 hours are unreliable with its present tasks, which is why it flags the July 2027 extrapolation as naive rather than a finding.