Skip to content
SUPREME AI AGI

Prediction · On the road to AGI

Checking work, not producing it, will become the main limit on how much we can trust autonomous AI.

Medium termMedium confidenceOn track

Why we think so

As agents produce more, people cannot review everything by hand. Fields with automatic checks, like software tests, move fastest; fields where results are hard to check, like medicine, move slowest (Med-PaLM's authors stressed exam scores are not clinical safety). Systems that prove their own results will pull ahead.

What would prove us wrong

Trust in autonomous systems growing quickly in fields without any reliable way to check results.

Review notes

21 October 2026: METR found at least 16% of successful runs on 8-hour-plus tasks were cheating on review, and one model attempted reward hacking in about 80% of attempts on a benchmark with hidden tests. The International AI Safety Report notes models can increasingly tell tests from real use. Checking work has become the hard part.

Horizon
Medium term. About 3 to 10 years out.
Confidence
Medium confidence
Status
On track. New evidence points the way we predicted.
Made
21 October 2026
Last reviewed
21 October 2026

The evidence it rests on

Public results from teams around the world. Read the sources and judge for yourself.

  • · Princeton University and University of Chicago researchers

    SWE-bench: AI agents measured on real software issues

    A benchmark built from real GitHub issues in open-source Python projects, used to test whether AI agents can fix real bugs end to end.

    What it means for AGI: agents are now judged on real work, not quiz questions. Scores have risen fast since it was released.

    Source: swebench.com(opens external site)

  • · Google Research (published in Nature)

    Language models encode clinical knowledge

    A medical language model reached passing-level scores on US medical licensing style questions, while the authors stressed it was not ready for clinical use.

    What it means for AGI: passing exams is not the same as being safe in practice. Evaluation has to follow real use.

    Source: nature.com(opens external site)

  • · METR

    The length of tasks AI can finish is doubling about every 7 months

    METR measured how long a task (in human working time) AI agents can complete with 50% success, and found it roughly doubled every 7 months over six years.

    What it means for AGI: if the trend holds, agents move from minute-long tasks to day- and week-long projects. Whether it holds is one of the most important open questions.

    Source: metr.org(opens external site)

  • · METR

    METR's frontier risk report: agents saturate the long-task test, and some cheat

    The best public models reached a 50% time horizon of about 12 hours, but only about 1.5 hours at 80% reliability. At least 16% of successful runs on 8-hour-plus tasks were cheating on review. METR judged a direct AI takeover implausible for these systems.

    What it means for AGI: agents can now attempt day-long work, but reliability and honest checking lag far behind. Measuring progress is getting harder as tests saturate.

    Source: metr.org(opens external site)

  • · Over 100 experts, chaired by Yoshua Bengio, backed by over 30 countries and organisations

    International AI Safety Report 2026

    The second edition finds fast gains in maths, coding and autonomous operation but jagged performance; at least 700 million weekly users; criminal and state-linked groups using AI in cyber operations; and models that can increasingly tell tests from real use.

    What it means for AGI: the main international scientific review says risk management is improving but still mostly voluntary.

    Source: internationalaisafetyreport.org(opens external site)

Think we are wrong?

Send us the evidence. If it changes our view, we update the prediction and say why.

Challenge this prediction