Instruction Tuning
Fine-tuning a base model on many (instruction, response) pairs so it learns to follow instructions instead of just completing text.
Instruction tuning is fine-tuning a pretrained base model on a broad dataset of (instruction, ideal response) pairs across many tasks, so it learns the general behavior of following instructions rather than merely continuing text. It is the step that turns a raw next-token predictor into a helpful assistant, usually before RLHF.
Worked example: a base model shown thousands of examples like “Summarize this: … → a summary” and “Translate to French: … → a translation” learns to respond to a NEW instruction it never saw, because it generalized the pattern of instruction-following. Gotcha: instruction tuning teaches the FORMAT and behavior of following instructions, not new facts — the knowledge came from pretraining — and its quality is bottlenecked by the diversity and quality of the instruction data; it is complementary to RLHF, which then aligns style and safety to human preferences.