How Researchers Measure Progress Toward AGI

AGI has a measurement problem before it has a finish-line problem. A model can set records in reasoning, coding or planning without proving that its competence transfers to unfamiliar work. The difficult evidence is not simply a harder exam; it is whether performance remains broad, adaptive and reliable when task-specific engineering stops doing most of the work.

Several 2026 research programmes attack the problem from different angles. Google DeepMind proposes levels of generality and performance. ARC Prize uses unfamiliar interactive environments. METR measures how task difficulty changes agent reliability over longer pieces of software work. Safety frameworks track capabilities that might become dangerous before anyone agrees that AGI exists.

For the bigger context, see our complete guide to AGI.

Why classic benchmarks are not enough

Benchmarks are valuable because they give researchers a repeatable test. But they create incentives. Once a benchmark becomes famous, developers can train on related data, optimise specifically for its format or build scaffolding around the test. Eventually, performance can rise without proving the underlying capability researchers originally cared about.

AGI makes this problem worse because the target is broad competence. A model that aces ten known exams may still fail badly when the task format changes.

Google DeepMind’s Levels of AGI framework

DeepMind’s framework separates two dimensions: performance and generality. Performance asks how well a system does relative to humans. Generality asks how broad the set of tasks is.

The value of the framework is conceptual clarity. It avoids treating a model that is superhuman at one task as “more AGI” than a system that is competent across many tasks. It also leaves room for different levels instead of demanding one binary threshold.

DeepMind’s 2026 cognitive taxonomy adds human baselines

In March 2026, Google DeepMind proposed a cognitive taxonomy for AGI evaluation. The approach recommends testing a broad suite of cognitive abilities, using held-out tasks to reduce contamination, collecting human baselines from a demographically representative adult sample and mapping each AI system against the human performance distribution.

The framework highlights learning, metacognition, attention, executive functions and social cognition as areas where better evaluations are especially needed. It matters because AGI is not only a problem of benchmark breadth; it is also a question of whether machines reproduce or surpass the cognitive functions that let people adapt across very different situations.

ARC-AGI-3 tests adaptation inside unknown environments

ARC Prize’s 2026 competition introduced ARC-AGI-3, an interactive benchmark made from game-like environments with no instructions, stated goals or known rules. An AI agent has to explore and infer how the environment works.

This is important because intelligence in the real world often begins before the problem has been neatly specified. A person entering a new software tool, game or workplace does not always receive a complete formal description. They experiment and build a model of what actions cause what outcomes.

ARC Prize reported that humans can solve the benchmark environments while frontier systems remained far behind at launch. The result should not be treated as a universal AGI score, but it exposes a class of weakness that conventional question-answer benchmarks can hide.

The benchmark then changed dramatically in September. GPT-6 Astra scored 62.7% with ARC Prize’s standard harness and 99.9% with a provider-adapter harness. ARC Prize says the result is major progress toward generalisation, but not proof of AGI. The gap between those two scores also shows why researchers need to report memory, context management, tools and scaffolding as part of an evaluation rather than treating the model name as the whole experimental condition.

METR measures how reliability changes as tasks get longer

METR’s time-horizon work asks how long a task would take a human expert and then estimates the task duration at which an AI agent has a given chance of success. The current suite focuses heavily on software engineering, machine learning and cybersecurity tasks.

The metric is useful because longer tasks create more opportunities for plans to fail. The agent must keep track of state, recover from mistakes, use tools and maintain coherence. These are capabilities likely to matter for more general autonomous systems.

METR also stresses the limits: the measure is about the difficulty of tasks in its suite, not literally how long a model can operate by itself.

Economic definitions produce a different measurement problem

OpenAI’s charter defines AGI around outperforming humans at most economically valuable work. That suggests measuring job tasks and economic substitution rather than only academic benchmarks.

But “most economically valuable work” is difficult to operationalise. Jobs are bundles of tasks, many of which involve trust, physical presence, relationships, responsibility or legal authority. An AI that performs the cognitive core of a task may not replace the human role around it.

Continual learning would be another important signal

Many current models are trained in large offline runs and then deployed. They can use context, tools and memory, but that is different from robustly acquiring new durable skills through experience the way humans do.

A stronger AGI evaluation would need to ask whether a system can learn a genuinely unfamiliar capability, retain it, transfer it and avoid damaging previously learned skills.

Embodiment complicates the definition further

Some researchers believe physical interaction with the world is important for general intelligence. Others think a system could qualify through cognitive and digital work alone.

This disagreement matters for measurement. If embodiment is required, software-only benchmarks can never be sufficient. If it is not, a highly capable digital agent could theoretically qualify without a robot body.

Safety thresholds are measurements too

Google DeepMind and Anthropic do not wait for an AGI label before measuring risky capability. Their frontier safety frameworks track domains such as cyber operations, biological misuse, harmful manipulation, model autonomy, misalignment and AI R&D acceleration.

These are not tests of “general intelligence” in the philosophical sense. They are operational tests of whether a model has become capable enough in a dangerous domain to require stronger security or deployment controls.

That makes them especially relevant for policymakers. Governance may need to respond to capability before science settles the AGI terminology.

Why one number is unlikely to settle the argument

Human intelligence is multidimensional. People can be excellent in one area, average in another and still function generally because they can learn, adapt and understand context. A machine measure faces the same complexity.

A credible AGI assessment would probably combine several forms of evidence: breadth, depth, transfer, learning efficiency, long-horizon planning, reliability, autonomy and perhaps embodiment. It would also need controls against benchmark contamination and task-specific scaffolding.

A credible AGI claim needs more than a company label

A company announcement that a model is “general” is not enough to settle the question. A serious AGI claim needs an explicit definition and, where possible, independent or reproducible evidence across more than one class of task.

High-risk capability is informative without being proof of AGI. TechnologyBlog’s OpenAI Astra cybersecurity coverage illustrates the distinction: strong performance in cybersecurity can matter enormously while still describing one capability domain rather than general intelligence.

The measurement question is the AGI question

Until researchers agree on what general intelligence means operationally, they cannot agree on one finish line. That does not make measurement pointless. It makes multiple, transparent measurements more important.

The strongest evidence will not be a single leaderboard score. It will be a pattern of robust performance across unfamiliar tasks, with fewer special cases and less task-specific engineering required to make the system succeed. For the dated record of what changed this year, see AGI Progress in 2026; for the evidence a convincing system would need, see 10 Capabilities an AGI Would Need to Demonstrate.