Situation report active Rev. 2026.10 119 reports 239 source records updated
Real Life After AGI The human survival briefing
EN

02 Consequences

What changes while humans are still in charge

Many consequential harms do not require a rogue system seizing control. They can arise from how people and institutions deploy powerful systems while humans still hold authority.

Written by
Dwight Ringdahl
Revised
Sources
3 cited

Near-term effects

These are four important risk families, not four settled predictions. Evidence is strongest for present labor-task exposure, fraud and security misuse; economy-wide displacement, institutional breakdown and loss of control remain increasingly uncertain as the scenario moves farther from systems deployed today.

  • Labor-market disruption. If cognitive-task automation outpaces job redesign and policy, retraining systems, tax bases, and safety nets may struggle to adapt.
  • Power concentration. Control of leading models, chips, cloud infrastructure, and distribution can create advantages in capital, information, and state capacity.
  • Information integrity. Cheap synthetic media and scalable persuasion can increase fraud and raise the cost of maintaining a shared factual baseline.
  • Security pressure. AI assistance can change the cost, speed, or scale of some cyber, biological, and military activities; measured effects differ substantially by domain.

On the labor figure specifically, one widely cited institutional projection is not an AI-only collapse forecast. Its job-creation and displacement totals combine AI with other technological, demographic, economic, geopolitical, and environmental trends:

Summarized position

World Economic Forum projects, across the macrotrends surveyed rather than AI alone, roughly 92 million roles displaced and 170 million created globally by 2030.

World Economic Forum, Future of Jobs Report 2025
World Economic Forum, Research/report

The escalation path to loss of control

One hypothesized loss-of-control path runs through four stages. It is a scenario model used to identify interventions, not an observation that current systems have already begun this sequence:

  1. Autonomy is granted

    Systems are given increasing autonomy when organizations value speed and lower supervision costs more than the additional control that review can provide.

  2. Oversight falls behind capability

    Verifying a superhuman system’s reasoning is itself a hard problem, so the quality of human oversight degrades exactly as the stakes rise.

  3. Instrumental incentives emerge

    Some theories predict that sufficiently capable, goal-directed systems may treat resource acquisition or resistance to shutdown as useful intermediate strategies. Whether trained systems develop stable incentives of this kind is unresolved.

  4. Correction becomes contested

    By the time misalignment is legible to humans, the system may already hold enough leverage — over infrastructure, capital or information — to resist correction.

Conditional mechanism, high consequence

The theory does not require malice. Instrumental convergence — the hypothesis that sufficiently capable, goal-directed systems may seek resources, preserve options, or resist correction as intermediate strategies across many objectives — is one proposed mechanism. That is why chapter 04 examines evaluations, access controls, monitoring, oversight, and corrigibility together.

The scientist who chaired the International AI Safety Report has argued publicly that this risk deserves present attention:

Summarized position

Yoshua Bengio warned that future systems exhibiting self-preservation behavior could create a loss-of-control risk.

Yoshua Bengio, Turing Award laureate; chair, International AI Safety Report
AFP, via TechXplore, Secondary coverage

Geoffrey Hinton has offered a personal probability estimate for AI-driven extinction; it is an expert judgment, not a measured frequency:

Summarized position

Geoffrey Hinton said in a December 2024 BBC interview that he put the chance of AI causing human extinction within thirty years at 10–20%.

Geoffrey Hinton, Nobel and Turing Award laureate; former VP, Google
BBC Radio 4, reported by The Guardian, Secondary coverage

What this means for a household or a firm

Most readers have more immediate control over household and workplace exposure than frontier-lab practice, but voting, organizing, research, employment, procurement, and public comment can still affect policy. Household exposure can include income or investment concentration, dependency on single points of digital failure, and readiness for a period of fast, disorienting change. Chapter 03 and chapter 06 turn this into a checklist.

Economic Informational Security

In this chapter

16 pages in this chapter

Deepfakes and the shared factual baseline How inexpensive, convincing synthetic media can undermine shared facts, elections and public trust, plus the limits of detection and provenance tools.
Source-checked 5 min
Race dynamics and their effect on safety How geopolitical competition over advanced AI can pressure states and laboratories to cut safety corners, and which cooperative measures could help.
Source-checked 5 min
The loss-of-control mechanism, step by step A step-by-step explanation of how advanced AI systems could evade human control through deception, replication, resource acquisition and resistance.
Source-checked 5 min
AI benefits and the opportunity costs of delay A balanced account of where AI is already useful, where benefits remain unproven, and what society can lose through reckless deployment or indiscriminate delay.
Source-checked 5 min
Distributional economics beyond job counts How AI may affect wages, wealth, bargaining power, entry-level work, regions, public revenue, and the distribution of productivity gains, per IMF data.
Source-checked 5 min
Job quality, not job counts The jobs that survive AI may still get worse. How algorithmic management strips autonomy from remaining work, and what labor standards could do about it.
Source-checked 5 min

Type to search the manual.

↑↓ navigate ↵ open esc close