Home Glossary Double Descent

Double Descent - Page 2

Double descent describes a model-performance curve that extends beyond the traditional bias–variance picture. As capacity grows, test error may initially decline, then worsen around the point where the model can fit the training data almost exactly, and finally improve again as the model becomes even more overparameterized. Similar behavior can appear as training time or dataset size changes. The phenomenon helps explain why very large neural networks can generalize despite having enough parameters to memorize their examples. It is not a guarantee that larger models will be better: data quality, optimization, regularization, architecture, noise, and evaluation design still determine whether the second descent occurs.