The recent debate around artificial intelligence has taken on an increasingly apocalyptic tone. Dario Amodei, CEO of Anthropic, has argued that frontier AI development should now be deliberately paced. His concern is not entirely theoretical: Anthropic’s latest threat-intelligence report documents increasingly autonomous cyber operations, AI-assisted weapons engineering, surveillance, influence operations, biological dual-use research, fraud, and illicit model distillation.
The full Anthropic report can be read here: Detecting and Countering Misuse of AI: September 2026 (PDF) .
Amodei’s argument is also laid out in his essay “We Must Pace the Frontier” .
The warning deserves serious attention. But there is another way to frame the problem.
What if slowing AI is itself an existential risk?
Humanity did not suddenly become vulnerable when large language models appeared. We were already extraordinarily vulnerable. We live almost entirely on one planet, depend on one biosphere, rely on tightly coupled technological systems, and still possess only limited capabilities for dealing with several classes of natural, technological, biological and geopolitical catastrophe.
The relevant comparison is therefore not:
AI risk versus no AI risk.
It is:
A world developing increasingly powerful AI versus a world already containing many catastrophic risks while developing the tools to solve them more slowly.
What Anthropic’s Own Report Actually Shows
Anthropic’s report is important precisely because it separates some real developments from more speculative future scenarios.
The company reports that AI is already changing the economics of cyber operations. In several cases, attackers used Claude not merely as a chatbot but as an engineering and operational layer capable of reconnaissance, exploitation, tool development, data processing and orchestration.
Anthropic describes operations in which multi-agent systems conducted reconnaissance, exploitation and data exfiltration while humans remained responsible primarily for selecting targets, setting objectives and reviewing results. See the official report: Anthropic Threat Intelligence Report, September 2026 .
One of the most important conclusions in the cyber section is that sophisticated attacks increasingly do not require equally sophisticated attackers.
Anthropic argues that AI has reduced the labor and expertise gap that previously separated state-sponsored operations from smaller groups and individual operators. Reconnaissance, software development, exploitation and stolen-data processing can increasingly be delegated to machines operating in parallel.
This is a serious change.
But it is not the same claim as saying that an autonomous AI has independently decided to conquer the internet or destroy humanity.
In Anthropic’s own case studies, humans still make many of the strategically important decisions: which targets to attack, what outcomes matter, what information to steal, and how to monetize or use the results.
The distinction matters.
“One human can now perform the work of a larger cyber team” is increasingly supported by evidence. “AI is becoming an independent civilization-ending strategic actor” remains a prediction.
The Biological-Risk Discussion Is More Nuanced Than the Headlines Suggest
The biological section of the Anthropic report is probably the most alarming at first glance. It describes researchers using frontier models in areas involving gain-of-function research, mammalian adaptation of avian influenza, orthopoxviruses, venoms and toxins.
Yet Anthropic itself is careful not to claim that Claude has already created an imminent biological catastrophe.
The company explicitly frames the cases as evidence that potentially dangerous dual-use research programs are interested in frontier AI and that some actors attempt to circumvent geographic and safety restrictions.
The relevant biological section and conclusions can be found in the official report: Anthropic — Detecting and Countering Misuse of AI .
In one avian-influenza case, Anthropic estimated that the benefit provided by the models was mainly clerical assistance, study ideation, data analysis and research-design support, rather than some magical leap from ordinary biology to instant bioweapon creation.
Biological weapons also did not begin with AI.
Advanced biotechnology, genetic engineering, pathogen research and laboratory automation already existed before today’s frontier models. AI may lower the barrier to some dangerous activities, and that deserves strong safeguards. But it may also lower the barrier to developing vaccines, antiviral drugs, antibodies, diagnostics and defensive countermeasures.
The real question is therefore not simply whether AI increases biological capability.
It is whether the net effect of that capability favors offense, defense, or some unstable combination of both.
AI Can Strengthen Attackers — but It Can Also Strengthen Defenders
A recurring weakness in some public discussions is the assumption that AI capability increases only one side of an arms race.
If an attacker can use an AI system to inspect a million lines of software for vulnerabilities, defenders can use AI to inspect those same lines for vulnerabilities.
If AI can generate malware faster, security systems can use increasingly capable models to identify malicious code, anomalous behavior and suspicious network activity.
If AI accelerates molecular design, it may accelerate both harmful research and the discovery of countermeasures.
If AI improves autonomous offensive systems, it can also improve early warning, interception, resilience and defensive automation.
This does not mean that offense and defense automatically cancel each other out. Some technological domains naturally favor attackers, while others favor defenders.
But saying “AI makes attackers stronger” without asking how much it makes defenders stronger is an incomplete analysis.
Anthropic’s own report actually illustrates this feedback loop. Threat actors develop new techniques with AI; Anthropic identifies those techniques, bans accounts, updates classifiers, strengthens safeguards and shares indicators with security partners.
The result is not a static world in which only criminals improve. It is an accelerating contest between increasingly AI-assisted offense and increasingly AI-assisted defense.
Humanity Already Lives With Many Other Catastrophic Risks
The focus on AI can create the psychological impression that humanity lived in relative safety until generative AI arrived.
That is not true.
We already face numerous catastrophic or potentially existential risks. Some are human-made, some are natural, and some are probably still unknown to us.
1. Large Asteroid or Comet Impact
Earth has been struck by large objects before and will be struck again. A sufficiently large asteroid or comet could devastate a continent, collapse global agriculture, or in an extreme case threaten civilization itself.
The interesting point is that this is one of the few natural catastrophes that technology can potentially prevent.
Humanity has already demonstrated asteroid deflection experimentally through NASA’s DART mission.
More advanced AI could improve sky surveys, trajectory prediction, autonomous spacecraft navigation, mission planning, materials engineering and interception systems.
2. Supervolcanic Eruption
A very large volcanic eruption could inject massive amounts of aerosols into the atmosphere, reduce incoming sunlight and severely disrupt agriculture across multiple continents.
The risk in any particular year is low, but low annual probability does not mean zero long-term risk.
AI could contribute to geological monitoring, magma-system modeling, atmospheric simulation, crop adaptation and contingency planning.
3. Global Nuclear War
Humanity already created civilization-scale destructive technology decades before ChatGPT.
A major nuclear exchange could kill millions immediately and potentially produce longer-term consequences through infrastructure collapse, food shortages and climatic effects.
No autonomous superintelligence is required for this scenario.
Human miscalculation, geopolitical escalation, technical malfunction or false warning could be sufficient.
4. A Natural Pandemic Worse Than COVID-19
COVID-19 demonstrated how vulnerable a globally interconnected civilization remains to infectious disease.
There is no biological law guaranteeing that the next globally transmissible pathogen must have relatively modest lethality.
AI-supported diagnostics, vaccine development, protein modeling, drug discovery, epidemiological modeling and automated laboratories could potentially reduce response times from years to months or even weeks.
5. Antimicrobial Resistance
Antibiotic resistance represents a slower-moving but extremely serious threat.
If large classes of antibiotics become ineffective, routine infections become more dangerous, while surgery, cancer treatment, organ transplantation and intensive-care medicine become substantially more difficult.
AI-assisted molecular discovery may become one of our strongest tools for finding new antibiotics and alternative therapies.
6. Extreme Solar Storm
A severe geomagnetic storm comparable to or larger than the Carrington Event would today encounter a civilization far more dependent on electronics than the world of the nineteenth century.
Satellites, communications, navigation systems, electrical grids, data centers and financial infrastructure could all be affected.
Better forecasting, grid design, automated recovery systems and resilient infrastructure could reduce such risks.
7. Cascading Failure of Global Infrastructure
Modern civilization is efficient partly because its systems are deeply interconnected.
Electricity depends on telecommunications. Telecommunications depend on electricity. Payments depend on digital networks. Logistics depend on software, fuel and communications. Water systems depend on power. Food distribution depends on transport and payments.
This creates the possibility of cascading failures in which one disruption propagates through several critical systems.
Advanced autonomous systems could also become part of the solution by detecting failures, isolating damaged components and restoring services much faster than human operators alone.
8. Global Food-System Failure
A single poor harvest is manageable.
Multiple synchronized agricultural failures caused by drought, volcanic cooling, crop disease, war, fertilizer shortages or energy disruption would be much harder to manage.
Humanity still depends on a surprisingly small number of staple crops and highly optimized supply chains.
Agricultural biotechnology, climate modeling, synthetic foods and automated controlled-environment farming could increase resilience.
9. Collapse of Important Ecosystems
Human civilization is technological, but it is not independent of biology.
Pollination, fisheries, freshwater systems, soil fertility and microorganisms remain essential components of our food and economic systems.
Large-scale ecological instability may not resemble a Hollywood apocalypse, but it could produce severe long-term economic and political disruption.
10. Abrupt Climate or Ocean-System Changes
Complex climate systems contain nonlinear processes, thresholds and feedback mechanisms.
Not every proposed tipping scenario is equally likely, but humanity still does not have perfect predictive control over large-scale atmospheric and oceanic systems.
Better scientific models are therefore not merely academic tools. They are part of civilizational risk management.
11. Hydrogen Sulfide and Black Sea Deoxygenation
The Black Sea represents a useful example of a large natural system whose chemistry remains potentially hazardous and scientifically complex.
Much of the deeper Black Sea is anoxic and contains dissolved hydrogen sulfide. The existence of this large sulfide-rich deep-water reservoir is well established.
Claims that the entire reservoir is certain to rise suddenly to the surface and ignite should be treated cautiously because such an extreme scenario is not established.
But the underlying issue is still real: changing stratification, oxygen depletion, chemistry and marine ecology can alter a system containing enormous quantities of reduced sulfur compounds.
The broader lesson is more important than any particular catastrophic prediction: we live next to planetary-scale chemical and ecological systems that we still do not completely understand or control.
12. Gamma-Ray Burst or Nearby Supernova
Some cosmic risks are extremely unlikely on human timescales but physically real.
A sufficiently nearby energetic astronomical event could damage the atmosphere and affect the biosphere.
These scenarios are not arguments for panic. They are reminders that Earth is not physically isolated from the wider universe.
13. Engineered Biological Threats Without AGI
Synthetic biology, gene editing, automated laboratories and DNA synthesis already create dual-use capabilities.
Advanced AI can increase those capabilities, but the underlying risk predates frontier models.
The rational response is therefore not to pretend the technology can be uninvented, but to invest aggressively in biosurveillance, rapid countermeasure development, secure laboratories and defensive biotechnology.
14. Laboratory Accidents
Not every catastrophic biological event requires a malicious actor.
Complex laboratories, dangerous pathogens and sophisticated biotechnology can produce risk through ordinary human error, procedural failure, equipment malfunction or incomplete understanding.
Better monitoring and AI-assisted laboratory safety could potentially reduce this category of risk.
15. Risks We Have Not Yet Conceptualized
This may be the most important category of all.
Every risk listed above is something the modern human mind already knows how to describe.
But there is no reason to assume that our current scientific civilization has discovered every important failure mode of complex technological societies, planetary systems or the universe.
Imagine an intelligent person living in 1700.
That person could understand war, famine, disease and volcanic eruptions.
But he could not meaningfully reason about nuclear chain reactions, global semiconductor infrastructure, engineered pathogens, coronal mass ejections affecting electrical grids, or software vulnerabilities disrupting an interconnected global economy.
The limitation was not stupidity.
The necessary scientific concepts had not yet been discovered.
There is no good reason to assume that humanity in 2026 has suddenly reached the point where every major future risk is conceptually visible to us.
The Largest Single Point of Failure: Earth Itself
There is another existential vulnerability that receives surprisingly little attention compared with AI doom scenarios.
Nearly all human civilization exists on one planet.
We have no independent human civilization on the Moon. We have no self-sustaining settlement on Mars. We do not have an orbital civilization capable of surviving indefinitely without Earth.
We store biological samples, seeds, genetic information and digital archives, but almost every meaningful continuation of human civilization still depends on Earth.
From an engineering perspective, this is a remarkable concentration of risk.
Humanity effectively operates one production environment with no fully independent off-site backup.
Becoming multiplanetary should therefore not be understood only as exploration, prestige or science fiction.
It can also be understood as civilizational redundancy.
Reaching that stage requires advances in robotics, energy, medicine, materials science, autonomous construction, closed-loop life-support systems, synthetic biology and spacecraft engineering.
These happen to be exactly the kinds of fields that powerful AI might accelerate.
The Opportunity Cost of Slowing AI
Discussions about AI safety usually calculate the risks created by more powerful AI.
Far less attention is given to the risks created by delaying capabilities that might solve existing problems.
Suppose advanced AI could reduce the time required to discover effective cancer treatments, antibiotics, antivirals or new energy technologies by several years.
A three-year delay in those capabilities is not morally or economically neutral.
During those years, people continue dying from cancer, cardiovascular disease, neurodegenerative conditions, infections and other diseases.
The same logic applies to planetary defense, climate adaptation, energy storage, fusion research, resilient agriculture and space infrastructure.
The correct risk equation is therefore more complicated than:
How dangerous is advanced AI?
It should look more like:
Risk created by AI
minus risk reduced by AI
plus risk accumulated while useful technologies are delayed.
We do not currently know the values of those three terms.
But any serious policy discussion should attempt to estimate all of them.
AI Development Has Moved Extremely Fast — but That Cuts Both Ways
Fifteen years ago, machine-learning systems struggled with tasks that today appear elementary.
A few years ago, language models were largely treated as systems that predicted text.
Today, frontier models write software, use tools, interact with computers, conduct research, coordinate subtasks and remain active on problems for increasingly long periods.
That rate of improvement absolutely justifies caution.
But it suggests something else as well.
Humanity may be approaching the first broadly useful technology capable of significantly increasing the rate at which scientific and technological progress itself occurs.
That possibility changes the safety calculation.
A technology that accelerates science may generate new risks, but it may also allow humanity to escape vulnerabilities that have remained unsolved for centuries.
We Should Also Examine the Incentives Behind the Safety Debate
None of this means that Dario Amodei or Anthropic must be acting in bad faith.
Their concerns may be completely sincere.
However, serious analysis should examine incentives even when a speaker is sincere.
Frontier AI laboratories occupy a highly unusual position.
They are simultaneously commercial companies, strategic geopolitical assets, potential regulatory targets and institutions asking society to trust them with increasingly powerful technology.
When the companies already closest to the technological frontier argue that developing frontier models should require exceptionally demanding safety procedures, two things can be true at once:
- Those procedures may genuinely improve public safety.
- Those procedures may also create enormous barriers to entry for future competitors.
Regulation that requires massive compliance teams, specialized security infrastructure, expensive evaluations and privileged relationships with governments will naturally be easier for existing multibillion-dollar frontier labs to satisfy than for a new entrant.
This does not invalidate regulation.
It means society should ask an additional question:
Who gets to define safe AI, and does the regulatory structure unintentionally lock today’s frontier companies into permanent dominance?
The Geopolitical Dimension
Anthropic’s report also spends considerable attention on illicit model distillation and attempts by Chinese AI organizations and intermediaries to extract capabilities from Claude.
The report describes alleged campaigns involving very large volumes of queries, fraudulent accounts, proxy networks and attempts to capture reasoning traces and training data.
These allegations and Anthropic’s evidence are detailed in the final section of the official report: Anthropic — Illicit Distillation and Scaled Abuse .
This helps explain why Amodei’s policy proposals are not limited to internal safety. They also involve protecting frontier-model capabilities, controlling access to compute and preventing strategic competitors from obtaining equivalent technology too quickly.
AI safety, industrial policy and geopolitics are therefore becoming intertwined.
That makes the debate much more complex than a simple question of whether AI is “dangerous.”
The Media Problem: Very Different Risks Are Being Merged Into One Word
Public coverage often compresses several distinct categories into the phrase “AI apocalypse.”
But these categories should be separated.
- Human misuse of AI: criminals, states or individuals using AI as a tool.
- Autonomous operational failure: agents behaving unexpectedly while pursuing an assigned objective.
- Loss of control over superhuman AI: a future system developing strategic objectives incompatible with human survival.
Evidence for the first category is already substantial.
Evidence for the second category is growing and deserves serious research.
The third category remains a future possibility rather than an observed real-world event.
Combining all three into one emotional narrative makes useful policy discussion harder.
A Better Alternative: Controlled Acceleration
The alternative to slowing frontier AI does not have to be uncontrolled development.
A more balanced strategy would be controlled acceleration.
That means developing AI rapidly in fields that increase humanity’s resilience, while applying unusually strong safeguards to the areas where autonomous capability creates the greatest asymmetric danger.
Humanity should aggressively accelerate AI applications in:
- medicine and drug discovery;
- antibiotic and antiviral development;
- cybersecurity and automated defense;
- planetary defense;
- energy generation and storage;
- materials science;
- climate and Earth-system modeling;
- agriculture and food resilience;
- robotics;
- space infrastructure;
- closed-loop life-support systems;
- AI alignment and control research.
At the same time, extremely strong controls may be justified around:
- autonomous weapons;
- high-risk pathogen engineering;
- mass surveillance;
- fully autonomous offensive cyber operations;
- unrestricted replication and self-propagation;
- systems capable of acting irreversibly without meaningful human oversight.
This approach does not assume AI is harmless.
It recognizes that technological capability is dangerous but also necessary.
The Present State of Humanity Is Not Particularly Safe
There is something strange about describing an AI agent that can work autonomously for several hours as the ultimate existential danger while humanity remains unable to maintain a self-sufficient settlement on another planet.
We cannot yet reliably redirect every dangerous asteroid.
We cannot guarantee rapid protection against every future pandemic.
We cannot prevent every nuclear escalation.
We cannot fully model every planetary tipping process.
We cannot operate a completely independent human civilization away from Earth.
In that sense, technological stagnation is not a safe baseline.
It is simply another risk profile.
Perhaps the Real Existential Risk Is Remaining Technologically Fragile for Too Long
Powerful AI may eventually become dangerous enough that humanity must control it extremely carefully.
That possibility should not be dismissed.
Anthropic’s own threat-intelligence report provides substantial evidence that increasingly capable models already lower the cost of harmful activity.
But a civilization that never becomes sufficiently technologically capable is also vulnerable.
Earth will continue to experience natural hazards. New pathogens will emerge. Human beings will continue to create weapons. Cosmic objects will continue crossing planetary orbits. Ecosystems will change. Unknown technologies will generate unknown risks.
And beyond all of the threats we already understand, there are almost certainly risks that our current scientific knowledge is still too primitive to recognize.
Our long-term survival therefore requires more than restraint.
It requires capability.
We need the capability to detect threats earlier. We need the capability to design countermeasures faster. We need the capability to repair complex infrastructure autonomously. We need the capability to develop new medicines rapidly. We need the capability to defend the planet. And ultimately, we need the capability to make human civilization geographically independent of a single planet.
The central question should therefore not be:
How do we prevent AI from becoming too powerful?
A better question is:
How do we become powerful enough to protect civilization from AI, from human misuse of technology, and from the physical universe at the same time?
That is a harder question.
But it is also a more complete one.
Perhaps the greatest existential risk is not that technological progress becomes too fast.
Perhaps it is that humanity remains technologically fragile for too long.
Sources and Further Reading
Anthropic — Detecting and Countering Misuse of AI: September 2026
https://www-cdn.anthropic.com/e50be2e51e7695dc4b1366a37a245a597377d3b5/Anthropic-Detecting-and-countering-091026.pdf
Dario Amodei — We Must Pace the Frontier
https://darioamodei.com/post/we-must-pace-the-frontier
Computable — Anthropic report on AI misuse
Anthropic-rapport over AI-misbruik slaat in als bom
VRT NWS — AI apocalypse discussion
VRT NWS: AI-apocalyps, bezorgdheid, Barack Obama and Dario Amodei
VRT NWS — Liveblog on warnings about an AI apocalypse
Waarschuwingen voor AI-apocalyps klinken steeds luider
Reacties
Een reactie posten