Engineered pandemics and bioweapons risk What AI-bio uplift studies show, what they do not show, and how information safeguards connect to real-world biodefense, including Anthropic's 2025 ASL-3 call.
Source-checked 5 min
07 Mechanisms
The mechanisms behind the worst-case outcomes
Chapter 02 covers the general loss-of-control mechanism. This chapter is the register of specific, named pathways researchers actually study — because the intervention that prevents one does not necessarily prevent another.
Why this isn't one scenario
"AI could kill everyone" compresses several structurally different failure modes into a single image. That compression is useful for a headline and useless for prevention, because the technical and policy interventions that address one mechanism may do little for another — controls on a large training run, for example, do not by themselves prevent misuse of an already-deployed model. More than 350 researchers and executives, including people affiliated with several major labs, signed a one-sentence statement in 2023 urging policymakers to treat extinction risk as a global priority. It records a position held by prominent signatories; it does not measure the probability of harm:
Signatories to the Statement on AI Risk hold that mitigating the risk of extinction from AI should be a global priority alongside pandemics and nuclear war.
safe.ai, Signed statement
Three broad families
The 2026 International AI Safety Report uses three broad families. Malicious use is a human deliberately directing AI capability toward mass harm — engineered pandemics, infrastructure attacks, or autonomous weapons. Malfunctions include reliability failures and prospective loss-of-control scenarios in which systems operate outside anyone's control. Systemic risks arise through widespread deployment and institutional interaction, including concentration, labor disruption, and gradual human disempowerment. Categories can overlap, and evidence strength differs across mechanisms. The chair of the report has publicly emphasized loss of control:
Yoshua Bengio warned that future systems exhibiting self-preservation behavior could create a loss-of-control risk.
AFP, via TechXplore, Secondary coverage
Geoffrey Hinton has also offered a personal probability estimate. It is expert judgment under uncertainty, not a measured frequency:
Geoffrey Hinton said in a December 2024 BBC interview that he put the chance of AI causing human extinction within thirty years at 10–20%.
BBC Radio 4, reported by The Guardian, Secondary coverage
What this chapter does not do
Editorial boundary
Each page distinguishes documented evidence from forecasts and theory while naming the mitigations researchers propose. None describes synthesis, exploit, or attack specifics — that information would not make a reader safer, and publishing it is not what informed preparation requires. See the sourcing standard for how consequential claims are classified and reviewed.
In this chapter
AI-enabled cyberattacks on infrastructure What AI is changing in cyber operations, why infrastructure is exposed, and why the offense-defense balance is uncertain, including a 2025 AI-espionage report.
Source-checked 5 min
Autonomous weapons and military escalation What autonomy means in weapons, what has been documented in conflict, and how AI may change escalation and accountability, as in Libya's disputed Kargu-2 case.
Source-checked 6 min
Gradual disempowerment A theoretical pathway in which accumulated dependence on AI erodes human control without a dramatic takeover, formalized across the economy, state, and culture.
Source-checked 5 min
Reward hacking and specification gaming How optimizers exploit imperfect objectives, what has been observed, and why present examples do not by themselves prove future loss of control.
Source-checked 5 min
Mass persuasion and epistemic manipulation What experiments show about AI persuasion, what real-world evidence does not yet establish, and how scale could change the risk, per a 76,977-person 2026 study.
Source-checked 5 min
Multipolar and multi-agent catastrophic dynamics How competition among states, companies and autonomous AI agents can create catastrophic outcomes even when every participant prefers to avoid disaster.
Source-checked 6 min
AI risk estimates and expert disagreement How to interpret surveys, forecasts, and probability estimates without manufacturing consensus or dismissing uncertainty, drawing on a 2,778-person survey.
Source-checked 6 min
AI, nuclear command, and crisis stability How AI decision support could affect nuclear crises even without launch authority, and which safeguards reduce escalation risk, like automation bias in crises.
Source-checked 6 min
Deception, sandbagging, self-replication, and exfiltration What four distinct frontier-model evaluations test, what laboratories have observed, and why none alone demonstrates catastrophic loss of control.
Source-checked 6 min
Food system cascade failures Nearly every extinction-adjacent scenario routes through food, not direct violence. How the food system fails in cascade, and what would actually break it.
Source-checked 5 min
Infrastructure fragility and single points of failure How much of modern life rests on components that take days, months, or years to replace — and why every catastrophic scenario acts on that same substrate.
Source-checked 5 min