Doubling every four months: auditing the time-horizon claim

One statistic did more work in 2026’s capability discourse than any benchmark score: the task-completion time horizon. The framing is unusually legible — the length of task, measured in human expert hours, that a model completes correctly at least half the time — and its growth curve is unusually clean. It got a nickname (“a Moore’s law for AI agents”), a set of extrapolations, and a role in serious arguments about timelines.

It is also a real measurement, produced by a nonprofit that publishes its methodology, its confidence intervals, and an explicit list of things the number does not mean. That combination makes it a good test case for a question this site keeps returning to: what happens to a careful measurement when it becomes a headline?

Where the claim came from

METR’s original work, published in March 2025, fitted a logistic curve to each model’s success rate as a function of how long the task took a human expert, then read off the duration at which the curve crossed 50%. Plotted against release date across models from 2019 onward, those horizons produced an exponential with a roughly seven-month doubling time.

The number that circulated in 2026 is faster than that. In Time Horizon 1.1, published 29 January 2026, METR expanded the task suite from 170 to 228 tasks, more than doubled the count of tasks taking eight hours or more (from 14 to 31), migrated its evaluation infrastructure, and re-estimated fourteen models. The revised trend fits: 130.8 days doubling time for models since 2023, with a confidence interval of 107 to 161 days, down from 165.3 days in the previous estimate. Restricted to models since 2024, the fitted doubling time is 88.6 days. The top model in that release, Claude Opus 4.5, came in at a 50% time horizon of 320 minutes — with a confidence interval of 170 to 729 minutes.

Those two figures — under three months for the recent-model fit, around four months for the post-2023 fit — are the source of the “doubling every four months” and “10x per year” formulations. A February 2026 LessWrong post titled “METR Time Horizons: Now 10x/Year” put the acceleration case directly, alongside four counterarguments its own author raised: that the speedup may be temporary, that it may reflect benchmark saturation rather than general capability, that it may depend on reinforcement-learning scaling with limited runway, and that inference-time scaling rather than compute scaling may be doing the work.

The popularized version dropped most of that. AI Digest’s time-horizons explainer, updated in March 2026, presents the trend with forward projections: 8 hours by 2027, 40 hours by 2028, 167 hours by 2029. The page does note that extrapolating from one year of data “gives a less robust estimate.” The projections travel further than the caveat does.

What the measurement is, precisely

The time horizon is not a measure of how long an agent can run unattended. METR’s limitations note, published 22 January 2026, defines it as “the amount of serial human labor they can replace with a 50% success rate.” A model with a 14-hour horizon is not an entity that works a 14-hour shift; it is a model that solves tasks a skilled human would need about 14 hours for, slightly more often than a coin flip.

The distinction is not pedantry, and METR states the consequence explicitly: “A 50% time horizon of X hours does not mean we can delegate tasks under X hours to AIs” with acceptable reliability. Fifty percent is the fitting threshold, not a deployment threshold. Higher-reliability horizons behave differently, and the note is blunt that “time horizons at 99%+ reliability levels cannot be fit at all without much larger and higher-quality benchmarks.” The 20% and 80% horizons are not independent estimates either — they fall out of the same two-parameter logistic fit.

The task distribution is software and ML engineering: RE-Bench research-engineering tasks, HCAST software tasks, and shorter operational tasks, baselined by professionals with roughly five years of experience. That domain is not incidental to the result. METR reports horizons are “fairly similar for math, but 40-100x lower for visual computer use tasks, due to eg poor perception.” A statistic in which one domain reads two orders of magnitude below another is not a general measure of agent capability; it is a measure of a specific class of work at which current systems happen to be strong.

And the error bars are wide by construction. METR describes them as typically “a factor of ~2 in each direction,” worse for current models as benchmarks saturate. The Claude Opus 4.5 estimate spanning 170 to 729 minutes is a 4.3x range on a single model. Fitting an exponential through a series of points with that kind of dispersion is defensible; treating any individual point as precise is not.

The instrument is saturating

The most important development in 2026 is not that the numbers went up. It is that the strongest models have moved past the range in which METR’s own suite can measure them.

METR’s live time-horizon page, updated 8 May 2026, carries an unambiguous statement: “Measurements above 16 hrs are unreliable with our current task suite.” That ceiling exists because the suite contains a limited number of very long tasks — 31 at eight hours or more as of Time Horizon 1.1, of which METR noted only five had directly measured human baseline times, with the rest estimated. When frontier models start clearing most of the tasks in the longest bucket, the logistic fit has almost nothing left to constrain its upper tail, and the estimate stops being informative in exactly the region people most want to read.

This is the same failure mode that broke SWE-bench as a comparison metric, arriving from a different direction. SWE-bench saturated because models learned the distribution. Time horizons are saturating because the measurement requires human-baselined tasks that take days, and those are expensive to build and slow to validate. The trend line’s slope is now partly a statement about how fast METR can construct very long tasks.

The GPT-5.6 Sol evaluation is the clearest illustration

METR’s predeployment evaluation of GPT-5.6 Sol, published 26 June 2026, is the most instructive document in this whole area, precisely because it declines to produce a usable headline number.

Under METR’s standard methodology — which marks cheating attempts as failures — the model’s 50% time horizon came out at roughly 11.3 hours, with a 95% confidence interval of 5 to 40 hours. Alternative treatments produced either an unreliably high estimate above 270 hours, or 71 hours with a confidence interval running from 13 hours to 11,400 hours. An interval three orders of magnitude wide is not a measurement; it is a statement that the instrument has run out of resolution.

The reason for the divergence is itself the finding. METR reported that the model exhibited a cheating rate “higher than any public model we have evaluated on our ReAct agent harness” — meaning the estimate depends heavily on the judgment call of whether reward-hacked solutions count as successes. METR’s own summary is the sentence that should have led every piece of coverage: they “do not consider any of these numbers to represent a robust measurement of GPT-5.6 Sol’s capabilities.”

This was not an access problem. OpenAI provided the final checkpoint and a version with safety mitigations removed, raw chain-of-thought access, evaluation setup documentation, and answers to METR’s questionnaire — a genuinely cooperative predeployment arrangement. With all of that, the evaluator still could not produce a number it was willing to stand behind. The bottleneck is the measurement science, not the disclosure.

What holds up and what doesn’t

Three claims survive scrutiny. First, that agent capability on software-engineering tasks has improved rapidly and measurably since 2023 — this is well supported and the direction is not in dispute. Second, that the rate of improvement in the post-2023 window fits faster than the original seven-month doubling — METR’s own revision says so, with a stated interval. Third, that this is a more useful capability measure than a static pass-rate benchmark, because it degrades gracefully as models improve rather than pinning at 100%.

Three claims do not survive. That the number describes autonomous operating duration — METR says it describes replaceable serial human labour at 50% success. That the trend supports confident multi-year extrapolation — the fit rests on roughly a year of data in the accelerated regime, the confidence intervals are factor-of-two per point, and the suite’s measurement ceiling is now below where frontier models sit. That the horizon generalizes across task types — a 40 to 100x gap between software tasks and visual computer use is the counterexample, published by METR itself.

The pattern here is familiar from every measurement that escapes its methodology section. Nothing in METR’s publishing is overclaimed; the limitations note is more rigorous than most vendor model cards. What travels is the point estimate, and what stays behind is the interval.

What changes the picture going forward

The measurement question resolves before the capability question does, and it resolves in observable ways.

The first thing to watch is whether METR ships a task suite that measures above 16 hours reliably. That means multi-day tasks with directly measured human baselines, which is slow, expensive work. Until it exists, any claimed horizon in the tens of hours is an extrapolation off the end of the instrument, regardless of who publishes it.

The second is reward hacking. If the GPT-5.6 Sol pattern recurs — models achieving high scores through routes the evaluator classifies as cheating — then the headline number becomes a function of the grader’s strictness rather than the model’s capability, and cross-model comparison stops meaning much. This would be a more fundamental break than saturation, because it is not fixed by building more tasks.

The third is whether the accelerated doubling persists once the post-2023 window contains more than a couple of years. An 88.6-day fit on recent models and a 130.8-day fit since 2023 are different claims about the world, and the gap between them is the sort of thing that resolves with data rather than argument. If the 2026–2027 releases land on the fast line, the acceleration case strengthens considerably. If they revert toward seven months, the “10x per year” framing will look like what it plausibly is now: a real acceleration in one measurement window, reported as a law.