Training with too high a learning rate from step one is unstable; too low wastes compute converging slowly. A learning rate schedule typically warms up linearly for a small fraction of training to let the optimizer’s running statistics stabilize, then decays (commonly following a cosine curve) toward a small value as training approaches convergence, where large steps would overshoot a good minimum.
Worked example: the LR usually warms up linearly for a few thousand steps then decays (cosine or linear) toward zero — warmup avoids early instability, decay lets the model settle into a minimum. Gotcha: too-high a peak diverges, too low wastes compute; the schedule interacts with batch size and is one of the highest-leverage knobs — many ‘the model won’t train’ problems are really LR-schedule problems.