Batch Size
How many training examples the model processes before each weight update.
Batch size is the number of examples used to compute one gradient before updating the weights. Larger batches give a smoother, less noisy gradient; smaller batches update more often and add regularizing noise.
Worked example: batch size interacts with the learning rate — a common heuristic is to raise the learning rate as you raise the batch size, since a bigger batch gives a more confident gradient you can step further along. Gotcha: huge batches can generalize worse and are capped by GPU memory, which is why gradient accumulation (sum gradients over several small batches before stepping) simulates a big batch without the memory cost.