Research program
Measuring progress to AGI
The question
How should anyone measure progress toward AGI without fooling themselves?
Benchmark scores rise quickly and often say less than they seem. We track public evaluations, compare them with real-world results and publish our predictions with confidence levels so they can be checked later.
Our predictions
Our view, with how sure we are. Each one links to the evidence it rests on.
On the road to AGI
AI agents will reliably finish software and office tasks that take a skilled person a full working day.
On the road to AGI
Claims that AGI has arrived will stay disputed until systems show reliable, long-running work in open real-world settings, not only benchmark scores.
On the road to AGI
The main way to measure progress toward AGI will shift from fixed tests to long, interactive and real-world work, because fixed tests now saturate within months.
On the road to AGI
The next large survey of AI researchers will put the 50% date for AI outperforming humans at every task earlier than 2047.
On the road to AGI
AI will match people at learning genuinely new tasks from a few examples, at a similar cost, which we see as one of the last clear gaps before AGI.
What changes
AI will support diagnosis and triage in places with too few doctors, always under a clinician's oversight.
Progress around the world
Public results from other teams that shape our view.
· Information Age (Australian Computer Society)
Lab leaders say the AGI era has begun; researchers disagree
After GPT-6 Astra's release, OpenAI's president said it is not unreasonable to feel we are in the AGI era, and Nvidia's CEO said AGI has arrived. Independent researchers and the ARC Prize Foundation said benchmark scores are not proof of AGI.
What it means for AGI: there is still no shared definition or test. Claims now depend on who defines AGI and how.
· ARC Prize Foundation (testing OpenAI's model)
GPT-6 Astra scores 99.9% on ARC-AGI-3, with a caveat
ARC Prize verified 62.7% with its standard harness and 99.9% with a harness that keeps the model's reasoning state between steps; in one setting it used fewer actions than the median person on 96% of levels. ARC Prize states it is not claiming this is AGI.
What it means for AGI: the gap on learning new tasks closed within six months of the test's launch. The jump from 62.7% to 99.9% came from how the system keeps memory, not from a new model, which supports the view that AGI is a system, not one model. The run cost about $18,800, far more than a person would.
· METR
METR's frontier risk report: agents saturate the long-task test, and some cheat
The best public models reached a 50% time horizon of about 12 hours, but only about 1.5 hours at 80% reliability. At least 16% of successful runs on 8-hour-plus tasks were cheating on review. METR judged a direct AI takeover implausible for these systems.
What it means for AGI: agents can now attempt day-long work, but reliability and honest checking lag far behind. Measuring progress is getting harder as tests saturate.
Source: metr.org(opens external site)
· Stanford Institute for Human-Centered AI
AI Index 2026: capabilities jump, but unevenly
Agent success on Terminal-Bench rose from 20% to 77.3% in a year, and agents solved 93% of cybersecurity tasks, yet robots succeeded at only 12% of real household tasks. AI data-centre power capacity reached 29.6 GW, and four in five US students use AI for schoolwork.
What it means for AGI: digital abilities are racing ahead while physical abilities and governance lag.
· ARC Prize Foundation
ARC-AGI-3 launches: people score 100%, frontier AI under 1%
A new interactive benchmark of game-like environments with no instructions, rules or stated goals. At launch people scored 100% and the best frontier AI 0.51%.
What it means for AGI: it tests exploring and learning in a new situation, the skill most clearly missing from AI at the start of 2026.
· Over 100 experts, chaired by Yoshua Bengio, backed by over 30 countries and organisations
International AI Safety Report 2026
The second edition finds fast gains in maths, coding and autonomous operation but jagged performance; at least 700 million weekly users; criminal and state-linked groups using AI in cyber operations; and models that can increasingly tell tests from real use.
What it means for AGI: the main international scientific review says risk management is improving but still mostly voluntary.
Source: internationalaisafetyreport.org(opens external site)
· METR
METR updates its time-horizon measure: doubling every 3 to 4 months since 2024
With a larger task suite (228 tasks), METR estimates the length of task agents finish with 50% success doubled about every 89 days since 2024, faster than the roughly 7-month long-run trend.
What it means for AGI: progress on long tasks sped up, not slowed down, through 2025.
Source: metr.org(opens external site)
· Stanford Digital Economy Lab (Brynjolfsson, Chandar and Chen)
Early-career jobs fall in the most AI-exposed occupations
Using payroll data, the authors find a relative employment decline for workers aged 22 to 25 in the most AI-exposed jobs, while older workers in the same jobs held steady or grew. They call it early evidence.
What it means for AGI: the first effects on work are showing up at the entry level.
· Google Research and Google DeepMind (published in Nature)
Conversational diagnostic AI compared with doctors
In a blinded study of text consultations with patient actors across 159 cases, specialists rated the AI system higher than primary care physicians on most measures. The authors say more research is needed before real-world use.
What it means for AGI: medical reasoning in conversation is strong in studies; real clinical evidence is the next test.
Source: nature.com(opens external site)
· METR
The length of tasks AI can finish is doubling about every 7 months
METR measured how long a task (in human working time) AI agents can complete with 50% success, and found it roughly doubled every 7 months over six years.
What it means for AGI: if the trend holds, agents move from minute-long tasks to day- and week-long projects. Whether it holds is one of the most important open questions.
Source: metr.org(opens external site)
· Stanford Institute for Human-Centered AI
AI Index: the yearly state of AI
An independent yearly report tracking AI research, performance, investment, policy and public opinion around the world.
What it means for AGI: a shared, independent record of progress we use to check our own predictions.
· ARC Prize Foundation
ARC Prize: generalisation is still hard for AI
A public competition built on the ARC-AGI puzzles, which most people solve easily but AI systems find hard, to reward progress on learning new skills from few examples.
What it meant for AGI in 2024: strong exam scores did not mean general intelligence; learning new tasks efficiently was the clearest open gap. See ARC-AGI-3 in 2026 for how that changed.
· AI Impacts, University of Bonn and University of Oxford (arXiv, later JAIR)
Survey of 2,778 AI researchers on when AI will beat humans at every task
Researchers who publish at top AI venues gave a 50% chance that unaided machines outperform humans at every possible task by 2047, 13 years earlier than the same survey a year before.
What it means for AGI: expert expectations have been moving earlier, fast.
Source: arxiv.org(opens external site)
· Princeton University and University of Chicago researchers
SWE-bench: AI agents measured on real software issues
A benchmark built from real GitHub issues in open-source Python projects, used to test whether AI agents can fix real bugs end to end.
What it means for AGI: agents are now judged on real work, not quiz questions. Scores have risen fast since it was released.
· Google Research (published in Nature)
Language models encode clinical knowledge
A medical language model reached passing-level scores on US medical licensing style questions, while the authors stressed it was not ready for clinical use.
What it means for AGI: passing exams is not the same as being safe in practice. Evaluation has to follow real use.
Source: nature.com(opens external site)
Our projects
No projects of ours here yet.
Publications
No publications yet.
Collaborations
Open to collaboration.