A scaling law is an empirical relationship
In machine learning, a scaling law describes how a measured quantity changes as an input grows. For large language models, researchers found that training loss often improves in a relatively smooth, predictable way as model size, training data, and computation increase. OpenAI’s 2020 work reported power-law relationships over tested ranges (Kaplan et al.). DeepMind’s later “Chinchilla” study showed that, under a fixed compute budget, many models had been trained on too little data and that parameters and training tokens should be scaled together more evenly (Hoffmann et al.).
These results were important because they made expensive training runs more predictable. A lab could run smaller experiments, estimate how loss would change at larger scale, and allocate a budget among data, parameters, and compute. But the law describes measurements inside an observed regime. It is not a physical guarantee that every capability will improve indefinitely or that enough graphics processors automatically produce AGI.
Compute is only one ingredient
“Compute” usually means the number of numerical operations used in training or inference, but even that shorthand hides important differences. Two training runs with the same nominal operations can produce different systems because of chip utilization, numerical precision, data quality, architecture, optimization, and software. Algorithmic improvements can obtain better performance from the same hardware. Fine-tuning, retrieval, tools, memory, and agent scaffolding can substantially change deployed behavior after pretraining.
Data matters in both quantity and composition. Duplicated, low-quality, or contaminated examples are not equivalent to carefully selected information. Synthetic data can extend or target a corpus, but errors may be amplified if generation and filtering are poor. Data rights, privacy, language coverage, and representation also influence what a model learns.
Inference-time computation adds another dimension. Some systems spend more operations generating and checking intermediate candidates for a difficult prompt. This can improve performance without changing the pretrained model, but it raises cost and latency and does not guarantee correctness. A complete scaling discussion now needs at least training compute, inference compute, data, algorithms, and the surrounding system.
Loss curves are not capability curves
Training loss measures predictive error averaged across enormous datasets. A smooth reduction in loss can accompany uneven changes on human-facing tasks. A benchmark may show a threshold because a latent improvement finally crosses its pass mark, because the model learns several required subskills, or because scoring converts gradual progress into a binary result. Other abilities may improve smoothly, remain flat, or regress after safety tuning or system changes.
This distinction prevents two opposite mistakes. The first is assuming that a predictable loss curve makes every future behavior predictable. It does not. The second is claiming that all capability gains are mysterious discontinuities. Many apparently abrupt benchmark gains become smoother when researchers use continuous metrics or more informative tests (Schaeffer, Miranda and Koyejo, 2023). Both uncertainty and partial predictability can be true.
Scaling also interacts with evaluation design. Public tests can saturate or leak into training data. A model may improve at test-taking without an equivalent gain in real-world reliability. Conversely, existing benchmarks may miss useful capabilities. Multiple evaluations, private tasks, human expert review, and deployment evidence are needed to interpret what the loss reduction bought.
Where the trend can bend
Physical capacity is not unlimited. Frontier training and serving depend on advanced chips, high-bandwidth memory, networking, electricity, cooling, land, construction, and supply chains. The International Energy Agency expects data-center electricity demand to rise substantially through 2030, while emphasizing uncertainty and strong geographic concentration (IEA, Energy and AI). Local grid delays can matter even when global energy is sufficient.
High-quality human-generated data is also finite, although the effect of that constraint depends on new data sources, multimodal training, synthetic data, and more efficient use of existing corpora. Economic limits may arrive before technical ones: a larger model must create enough value to justify training, serving, and opportunity costs. Latency and memory bandwidth can constrain products even when aggregate operations are available.
Diminishing returns are built into scaling. A power law may continue while each additional improvement requires much more expenditure. That does not mean progress stops, but it changes the economics. A new architecture or algorithm can move the curve, while safety, reliability, or embodiment may require advances not captured by next-token prediction loss. Nobody can establish from current scaling laws alone where those breakthroughs will occur.
Why compute remains a governance lever
Despite these caveats, advanced compute is unusually physical and concentrated. Cutting-edge chips are manufactured through a narrow global supply chain, and the largest training clusters require visible capital, power, and networking. Governments therefore use chip export controls, reporting requirements, and data-center rules as proxies for frontier capability. Compute can be easier to monitor than algorithms or copied software.
The proxy is imperfect. Thresholds can become obsolete as algorithms grow more efficient. Distributed training and inference complicate accounting. A compute rule can burden legitimate research, entrench large incumbents, or encourage avoidance if written badly. Training compute also misses capable systems assembled through fine-tuning, tool use, or many inference calls. Effective policy should update thresholds, account for efficiency, protect privacy, and focus obligations on risk rather than treating operations as harm.
Compute governance is strongest when paired with capability evaluation. Compute can identify runs worth scrutiny; evaluations can test whether a resulting system crosses a dangerous-capability threshold; deployment monitoring can reveal misuse and failures. No one metric should bear the entire regulatory load.
Scaling does not settle AGI timelines
Rapid investment is evidence that developers expect returns from larger and more efficient systems. Historical scaling is evidence that additional resources have often improved measured performance. Neither proves a date for AGI. Forecasts must model hardware, energy, data, algorithms, inference, reliability, and the definition of AGI itself.
Claims that “scaling has stopped” require specifying which metric and system. Claims that “scaling is all you need” require showing that the remaining gaps are reducible to the same trend. Between those slogans lies the defensible position: scaling laws are among the most useful empirical regularities in modern AI, but they predict averages within regimes better than they predict qualitative social consequences.
What readers should ask
When a new model is attributed to scale, ask which resource increased, by how much, and against what baseline. Was the comparison compute-optimal? Did data or algorithms change? Is the reported gain on training loss, a benchmark, expert work, or a deployed outcome? What did reliability and cost look like? Are results independently reproducible?
Those questions preserve the real lesson of scaling laws without turning it into mythology. More compute has repeatedly enabled better models. It is a major industrial input and a practical point of oversight. It is not a universal unit of intelligence, a guarantee of safe behavior, or a countdown to AGI.