Transfer Learning
Reusing a model pretrained on a big general task as the starting point for a specific one — the foundation of modern AI.
Transfer learning is taking a model trained on a large general task and reusing its learned representations as the starting point for a different, usually smaller, task — instead of training from scratch. It is the paradigm behind foundation models: pretrain once on vast data, adapt many times.
Worked example: a vision model pretrained on millions of images already “knows” edges, textures, and shapes, so fine-tuning it on a few thousand medical scans works far better than training a fresh model on those scans alone. Gotcha: transfer works when source and target domains share structure — the further the target from pretraining, the less transfers — and full fine-tuning risks catastrophic forgetting, which is why lightweight adaptation (LoRA, freezing early layers) is often preferred.