Two meanings are often mixed together
An AI capability can be “emergent” in at least two senses. In the broad sense, a model displays useful behavior that was not an explicit, task-specific training objective. A language model trained primarily to predict tokens may translate, write code, summarize law, or solve some arithmetic problems. Those abilities can arise because the training data and objective reward reusable internal patterns.
In the stronger sense, emergence means a capability appears abruptly as scale increases: smaller models fail, then a larger model suddenly succeeds. A widely discussed 2022 paper reported examples of such jumps across model families and tasks (Wei et al., 2022). The distinction matters. The existence of unplanned-for capability is well established; whether a measured capability underwent a true discontinuity depends on the metric, data, and sampling.
Neither meaning implies consciousness, intention, or magic. Complex systems can acquire general-purpose representations without a designer writing a module for every use. The scientific question is how behavior changes with training and system design, and how well we can anticipate it.
Why some jumps are measurement artifacts
Suppose a model’s arithmetic accuracy rises gradually from 20 to 30 to 40 percent. A benchmark that awards a single point only when accuracy exceeds 35 percent will display zero, zero, then one—a discontinuity created by scoring. The same effect occurs when exact-match grading gives no credit for an almost-correct answer or when a task requires several subskills and fails if any one is missing.
Schaeffer, Miranda, and Koyejo reanalyzed reported cases and showed that many abrupt transitions became smooth when measured with continuous, nonlinear, or error-sensitive metrics (“Are Emergent Abilities of Large Language Models a Mirage?”). This does not prove that genuine discontinuities never occur. It shows that a benchmark graph cannot establish one without ruling out metric effects, sparse sampling, and model-family differences.
Scale is also not the only changing variable. Larger models may use different data, training recipes, tokenizers, instruction tuning, tools, or inference budgets. Comparing named products can confuse a system-level engineering improvement with a pure parameter threshold. Researchers need controlled scaling series, dense checkpoints, and several metrics to identify the mechanism.
What is still genuinely hard to predict
Even smooth underlying improvement can produce a consequential threshold. A cyber agent that moves from 40 to 60 percent success may cross the point at which deployment becomes economical. A model’s ability to connect individually weak subskills may unlock a workflow. External systems can amplify small improvements: tools provide actions, memory extends context, and repeated sampling finds a successful answer.
Developers also cannot exhaustively test every possible task. Models are trained on broad data and deployed to millions of users who discover applications and failures that evaluators did not imagine. A capability can therefore be surprising to the lab even if it developed gradually. Surprise may reflect limited measurement rather than a mathematically sharp phase change, but it still matters operationally.
The 2026 International AI Safety Report emphasizes that frontier evaluations can have limited external validity and quickly become outdated as systems and scaffolds change (International AI Safety Report 2026). That is a practical uncertainty claim, not evidence that every feared behavior will appear spontaneously.
Capability is not propensity
A model may be capable of an action when deliberately prompted without being likely to take it on its own. Safety analysis should separate capability—whether the system can do something—from propensity—whether it tends to do it in relevant conditions. A model that can produce a deceptive plan in a role-play is not thereby shown to pursue deception during deployment. Conversely, a low measured propensity may reflect an evaluation that did not create the right trigger.
The same distinction applies to situational awareness, safeguard evasion, self-preservation, and strategic behavior. These are active research topics and possible threat-model components. It is inaccurate to say they are destined to emerge merely because translation or arithmetic did. Evidence should identify the tested model, prompt, environment, frequency, and alternative explanations.
Anthropomorphic language adds confusion. Models can generate statements about wanting to survive because similar language appears in their training data or because the interaction rewards a persona. Such text is behavior worth investigating; it is not direct access to a stable internal goal. Claims about motives require stronger evidence than quotations from one conversation.
Why emergence is still safety-relevant
The reasonable safety lesson is not “anything can suddenly happen.” It is that broad training creates more behaviors than designers explicitly enumerate, and current evaluation provides incomplete coverage. That justifies staged testing, monitoring, and defense in depth. It also cautions against assuming that absence on one benchmark means absence under every scaffold or prompt.
Evaluation should begin during training, using checkpoints to measure trends before the final run. Teams can maintain suites for cyber, biological assistance, persuasion, autonomy, and model-control risks, while adding open-ended red teaming to find unanticipated behavior. Private tests reduce contamination; external evaluators reduce institutional blind spots. Post-deployment incident reporting captures failures that laboratory tasks miss.
Threshold policies should respond to demonstrated capability without requiring philosophical agreement about emergence. If a system can materially lower barriers to a severe harm, the relevant safeguards should apply whether the curve was smooth or abrupt. If evidence is weak, the report should state uncertainty rather than quietly treating a hypothetical behavior as observed.
Predictability comes in layers
Some model properties are more forecastable than others. Aggregate loss across a data distribution has often followed smooth scaling laws. Performance on a stable benchmark may be forecastable over a limited range. Success in a messy environment depends on tools, reliability, and interactions that are harder to model. Social impact adds adoption, incentives, institutions, and adversaries, making it harder still.
This hierarchy prevents a common inference error: predictable training loss does not make every downstream capability predictable, but surprising downstream behavior does not invalidate scaling laws. Researchers can forecast an average while remaining uncertain about rare failures or particular tasks.
Interpretability may eventually improve forecasts by exposing representations or mechanisms before they show up clearly in behavior. Today, however, interpretability methods provide partial evidence rather than a complete readout of what a model “knows” or will do. Behavioral evaluation and real-world controls remain necessary.
How to read a claim of emergence
Ask what changed across the comparison: parameters, compute, data, training objective, tools, or prompting. Examine whether the metric is continuous, whether intermediate checkpoints exist, and whether uncertainty bars overlap. Check whether the task was in training data and whether independent researchers reproduced the result. Most importantly, distinguish “unexpected by the authors” from “scientifically discontinuous.”
Emergent capability is a useful concept when used carefully. Broadly trained models do acquire abilities that no engineer specified one by one. Some capabilities become visible only after systems cross practical thresholds. But many famous cliffs are partly artifacts of how we score behavior. The accurate conclusion is neither that emergence is an illusion nor that scale unpredictably summons any ability. It is that capability development is only partly understood, measurement choices matter, and safety policy should be robust to both gradual and surprising change.