We are building machines that can outperform us at some tasks. The difficult question is how far that advantage can go, and whether we will know when we reach the limit.
The most unsettling possibility about artificial intelligence is not that it will suddenly become conscious, develop a personality, or announce that it has surpassed humanity.
It is that it might become extraordinarily capable at solving problems we do not fully understand, while we are still arguing about what intelligence actually means.
That is where the science of superintelligence begins. Not with science fiction, but with measurable capabilities, experiments, mathematical assumptions, engineering bottlenecks, and a future that nobody can predict with complete confidence.
And it raises a surprisingly important question: if today's AI systems can reason, write code, analyze data, and assist scientific research, why do we still call them artificial intelligence instead of simply calling them software?
The answer reveals something important about both the technology we have built and the technology we might build next.
intelligence is not one thing
We tend to treat intelligence as a single quality, even though real life keeps proving otherwise.
A mathematician can solve problems that leave most people confused and still struggle to understand another person's emotions. A gifted engineer can design an elegant system and overlook an obvious security flaw. Human intelligence is powerful, but it is uneven.
Machines are no different in an important respect. They can excel at one task while failing at another that appears much simpler.
In its International AI Safety Report 2026, an international group of more than 100 experts described this unevenness as a defining feature of general-purpose AI. Leading systems can solve advanced mathematical problems, write software, and assist scientific research, yet still produce false information or make basic reasoning mistakes.
Source: International AI Safety Report 2026
This matters because intelligence cannot be measured by one impressive demonstration.
A machine winning a chess championship tells us something about its ability to play chess. It does not establish that the machine can conduct medical research, negotiate a peace agreement, or understand the consequences of its decisions.
The distinction becomes even more important when we discuss superintelligence.
Philosopher Nick Bostrom described superintelligence as an intellect that vastly outperforms the best human brains across practically every field, including scientific creativity, general wisdom, and social skills. This is a much stronger claim than saying a computer is better than a person at a particular task.
Source: Nick Bostrom, How Long Before Superintelligence?
The real test is not whether a machine can beat us at something. It is whether it can outperform us across almost everything that requires intelligence.
And we are nowhere near having a universally accepted test that can settle that question.
the gap between today's AI and superintelligence
Every new AI breakthrough creates the same temptation: to mistake a remarkable capability for a finished destination.
Today's systems have achieved results that would have seemed extraordinary just a few years ago. They can generate working code, analyze complex documents, solve difficult scientific questions, and help researchers explore ideas that would otherwise require substantial time and effort.
But general-purpose does not mean universally competent.
A general-purpose AI system can handle many categories of work without being equally reliable across all of them. Its performance depends on the task, the information available, the tools it can access, and the conditions under which it operates.
The 2026 International AI Safety Report highlights this gap between performance in controlled evaluations and usefulness in the real world. A model might perform exceptionally well on a benchmark yet struggle with unfamiliar interfaces, unexpected constraints, or a long project where small mistakes accumulate.
Think about asking an AI system to build an application. Producing a promising first version is one achievement. Maintaining it, detecting subtle security vulnerabilities, handling unusual user behavior, and keeping it reliable over months is another.
The difference is between solving a problem once and being dependable across an entire process.
There is also no scientific requirement that a superintelligent system must be conscious. Intelligence, in the sense used in many superintelligence discussions, concerns cognitive capabilities. Whether a machine has subjective experiences is a separate, unresolved question.
Nor does superior intelligence automatically imply wisdom, benevolence, or good judgment. A system could become extraordinarily capable at achieving a goal without that goal reflecting human values.
That possibility is precisely why researchers distinguish capability from alignment, the problem of making AI behavior reliably consistent with intended goals and acceptable constraints.
We should therefore resist two equally misleading conclusions: that today's AI is merely autocomplete with a better interface, or that every impressive result proves superintelligence is arriving tomorrow.
The evidence supports a more interesting position. AI capabilities are advancing, but the distance between present systems and broadly superior, dependable intelligence remains uncertain.
the experiment that makes the future harder to dismiss
One of the most revealing questions in AI research is not how well a model answers a question. It is how long it can keep working on a difficult task without falling apart.
Researchers at METR, an organization that evaluates frontier AI capabilities, have studied this through software tasks with measurable outcomes. Their task-completion time horizon estimates the length of tasks an AI agent can complete at a specified success rate, using human expert completion time as the reference.
The distinction is important. The measure is not simply how many lines of code a model produces or how quickly it responds. It estimates the duration of tasks that an agent can complete with a given probability of success. METR's methodology also has limitations, including the types of tasks represented and the reliability of estimates for longer tasks.
Source: METR, Task-Completion Time Horizons
Why does this matter for superintelligence?
Because many economically important activities are not single questions. They are sequences of dependent decisions: investigate a problem, develop a solution, test it, discover a failure, revise the approach, and verify the result.
If AI agents become more reliable at longer tasks, they could take responsibility for increasingly substantial pieces of work rather than merely assisting with individual steps.
But there is a catch. A trend in task duration does not prove that AI will eventually automate every profession, and a longer benchmark horizon does not mean an agent can independently complete every real-world project of that length.
The environment matters. So do tools, error recovery, evaluation methods, and the difference between a clearly specified coding challenge and an open-ended business or scientific problem.
Still, the research gives us something more useful than a dramatic prediction: a measurable way to investigate whether AI systems are becoming capable of sustained work.
The important question is no longer only what AI can do. It is how much work we can reliably trust it to finish.
That question leads directly to an even more consequential experiment: can AI perform the research needed to improve AI itself?
what happens when AI helps build better AI?
Most technologies improve because people improve them. Engineers design better chips, researchers develop new training methods, and teams discover ways to make systems more efficient.
AI could increasingly participate in that process.
A model that writes code, analyzes experimental results, proposes model architectures, or identifies bugs can help researchers work faster. If these contributions improve the next generation of AI, which then contributes to further improvements, development could become more efficient.
This is the basic idea behind AI-assisted AI research. In stronger versions of the argument, increasingly capable systems could automate substantial portions of the research process, potentially accelerating future progress.
But three claims must not be confused.
First, AI can assist AI research. Second, AI can automate a significant share of AI research. Third, AI can autonomously improve its own capabilities in a way that produces sustained, accelerating progress. Each claim is stronger than the previous one.
Evidence for the first does not automatically establish the third.
A useful example comes from RE-Bench, a research engineering benchmark published at the 2025 International Conference on Machine Learning. It compared AI agents with human experts on seven challenging machine-learning research engineering environments.
In the researchers' experiments, the strongest AI agents scored around four times as highly as human experts when both received a total time budget of two hours per environment. Yet humans performed better when given substantially more time, achieving about twice the score of the strongest agents in the 32-hour comparison. These findings apply to the benchmark's particular tasks and experimental setup, not to AI research as a whole.
Source: Wijk et al., RE-Bench, ICML 2025
The result is fascinating precisely because it cuts both ways. AI agents can make rapid progress on certain research engineering problems, but that does not establish that they can independently direct an entire research program.
A successful experiment must still be interpreted. A promising result must survive replication. A proposed improvement must work outside the conditions in which it was discovered.
Even if AI automates more research, progress could remain limited by computing resources, energy, high-quality data, hardware production, experimental bottlenecks, or diminishing returns from existing methods.
An intelligence explosion is therefore a possible scenario, not an observed scientific fact.
The serious question is whether AI-assisted research eventually becomes sufficiently capable, reliable, and autonomous to change the speed of AI development itself.
That is something researchers can investigate. It is not something we should announce as inevitable.
intelligence becomes more interesting when it leaves the chatbot
Some of the most convincing demonstrations of AI's usefulness do not involve a machine pretending to be a human.
They involve solving a problem that humans have struggled to solve efficiently.
Consider weather forecasting. In 2023, researchers published GraphCast in Science, describing a machine-learning system that predicts global weather conditions up to ten days ahead. In its evaluation against the European Centre for Medium-Range Weather Forecasts' high-resolution deterministic forecasting system, GraphCast outperformed it on approximately 90% of 1,380 evaluated forecast targets. It could generate a forecast in under a minute on the reported hardware.
Source: Lam et al., Science, 2023
This was not evidence of a machine possessing universal intelligence. It was evidence that machine learning could produce highly capable results in a demanding scientific domain.
That distinction does not diminish the achievement. It explains why it matters.
Weather forecasting combines enormous datasets, complicated physical processes, and decisions that affect agriculture, transport, energy, and disaster preparedness. Better predictions can have practical value even when the system is not intelligent in every human sense.
AI can also assist scientists with literature reviews, data analysis, hypothesis generation, and experimental design. Yet an AI-generated hypothesis is not a confirmed discovery, and a convincing explanation is not a substitute for experimental evidence.
The pattern is worth noticing: AI's usefulness does not depend on proving that it thinks exactly like a human.
A system can be valuable because it identifies a pattern we missed, searches a space of possibilities more efficiently, or performs a calculation at a scale that would otherwise be impractical.
This is why dismissing AI as a meaningless label is as mistaken as treating every AI product as a step toward superintelligence.
The term describes a broad family of computational approaches and systems, not a promise that every system possesses human-like understanding.
And it leaves room for something genuinely important: machines can extend human capabilities without first becoming superior to humans at everything.
why the word "AI" still earns its place
The phrase artificial intelligence has been used for decades. It covers systems that differ enormously in their design, capabilities, and limitations.
Some AI systems classify images. Others predict weather, translate languages, recommend content, generate text, or control robots. Modern general-purpose models can support multiple tasks, while specialized systems remain useful for narrower problems.
What connects these applications is not that they all possess human consciousness. It is that they use computational methods to perform tasks associated with capabilities such as learning, perception, prediction, language, planning, or problem-solving.
The term is broad, but broad does not mean useless.
Medicine is also a broad word. It does not imply that every treatment works, every diagnosis is correct, or every scientific question has been answered. It names a field whose methods and capabilities must be evaluated individually.
AI deserves the same discipline.
When a company claims its system can reason, we should ask what reasoning tasks it can solve and how that performance was measured. When a product claims autonomy, we should ask which actions it can take, under what constraints, and how it recovers from failure.
When someone predicts superintelligence, we should ask what definition they are using, what evidence supports the prediction, and what assumptions connect current capabilities to the proposed future.
Changing the name does not answer any of these questions.
Nor should the term AI be treated as evidence of consciousness. A system's ability to produce convincing language does not, by itself, establish subjective experience. Researchers still lack a universally accepted scientific test that settles whether current AI systems are conscious.
The most useful approach is to separate three things: what a system demonstrably does, what researchers believe explains its behavior, and what people speculate it might eventually become.
The first can be tested directly. The second requires scientific investigation. The third requires explicit uncertainty.
AI is a useful word when it names a field of real capabilities. It becomes misleading when we use it as a substitute for evidence.
the problem nobody can solve with a benchmark alone
The more capable a machine becomes, the more important it becomes to understand what happens when it makes a mistake.
A chatbot that produces an incorrect answer can mislead a reader. An AI agent connected to software, financial systems, or scientific infrastructure may be able to take actions that create much larger consequences.
This is why AI safety research examines more than raw intelligence. It also investigates reliability, cybersecurity, misuse, human oversight, and whether systems remain within their intended constraints.
The 2026 International AI Safety Report identifies important gaps in current understanding. Researchers cannot reliably predict every capability a model will develop, and evaluations do not always reveal how a system will behave after deployment. Existing safeguards can reduce risks without eliminating them.
Superintelligence raises a harder version of the same problem. If a future system could outperform humans across many cognitive domains, how would we verify that it would consistently respect human intentions?
There is no established, universally accepted solution to that hypothetical challenge. Researchers investigate alignment techniques, interpretability, evaluations, and control methods, but their effectiveness against substantially more capable future systems remains uncertain.
We should neither assume disaster is inevitable nor treat the absence of a demonstrated catastrophe as proof of safety.
The sensible response is to test systems before trusting them with consequential decisions, improve independent evaluations, and make safeguards proportionate to the capabilities being deployed.
what evidence would actually convince us?
Predictions about superintelligence often depend on assumptions about future progress. Scientific progress requires something more demanding: observations that can distinguish competing explanations.
Evidence for increasingly general intelligence would include reliable performance across diverse, unfamiliar tasks, sustained work over longer periods, effective recovery from unexpected failures, and the ability to transfer knowledge to situations unlike those encountered during training.
Evidence for automated AI research would require showing that systems can contribute meaningfully across the research process, not merely produce plausible ideas or isolated coding improvements.
Evidence for an intelligence explosion would require demonstrating that AI-driven improvements reliably accelerate subsequent improvements enough to overcome practical bottlenecks.
No single benchmark can establish all of this. Independent replication, real-world testing, transparent methods, and careful comparisons with human performance would be essential.
Even then, evidence of exceptional capability would not automatically establish consciousness, wisdom, or safety.
The future remains uncertain. That uncertainty is not a reason to stop investigating. It is a reason to be precise about what we know.
conclusion: the future does not need exaggeration
Superintelligence is a serious scientific question, but it is not an established description of today's AI.
We have evidence of systems becoming more capable, assisting research, solving difficult problems, and taking on increasingly complex tasks. We also have evidence of persistent failures, uneven performance, and substantial gaps in our understanding of their behavior.
The honest position is neither blind optimism nor automatic fear. It is to measure what machines can do, investigate what they cannot, and prepare for plausible futures without pretending that predictions are facts.
AI remains a useful word because it describes a real and evolving field. Superintelligence gives us a way to investigate what might happen if its capabilities go much further.
The distinction between those ideas matters more than ever.
"The future of intelligence will be decided not by what we call our machines, but by what we can prove they are capable of doing."
sources and further reading
The article draws on published research and scientific assessments, with predictions distinguished from demonstrated results.
- International AI Safety Report 2026, led by Yoshua Bengio. Read the report. Evidence on capabilities, limitations, risks, and uncertainty.
- METR, Task-Completion Time Horizons. Read the methodology and results. Methods for measuring AI agents' performance on tasks of different durations.
- Wijk et al., RE-Bench, ICML 2025. Read the research paper. A controlled comparison of AI agents and human experts on machine-learning research engineering tasks.
- Lam et al., GraphCast, Science, 2023. Read the peer-reviewed paper. Research on machine-learning-based global weather forecasting.
- Nick Bostrom, How Long Before Superintelligence? Read the paper. Background on the concept and definition of superintelligence.