Scaling Laws

Scaling laws are empirical mathematical relationships that estimate how an AI model’s loss or capability changes as training compute, dataset size, or parameter count increases. Researchers use them to plan experiments, allocate resources, and forecast whether a larger training run is likely to deliver worthwhile improvement. The relationships often follow smooth power-law trends within a tested range, but they are not universal laws of intelligence. Architecture changes, data quality, evaluation choice, optimization, inference-time computation, and capability thresholds can alter the curve. Extrapolating far beyond observed evidence is risky, so scaling forecasts should include uncertainty and be revised as new model families and training methods appear.