Fine-Tuning
Continuing to train a pretrained model on your data to specialize its behavior.
Fine-tuning adapts a base model to a task, domain, or style by training further on curated examples. Parameter-efficient methods like LoRA update only small adapter weights, making it cheap. Often compared against prompting and RAG as ways to change model behavior.
Worked example: full fine-tuning updates every weight (expensive, big checkpoints); LoRA trains well under 1% of parameters as small adapter matrices and ships a megabyte-scale file, while the frozen base does the heavy lifting. Gotcha: fine-tuning is excellent for teaching behavior, format, and style and a poor way to inject facts — baked-in knowledge goes stale, is hard to update, and can cause catastrophic forgetting. Rule of thumb: RAG for knowledge, fine-tuning for behavior.