Transfer learning

Definition

Using knowledge learned from one task or dataset to improve performance on a different but related task. Transfer learning is why pre-trained foundation models can be fine-tuned for specialized applications.

Why it matters

Transfer learning is the economic foundation of the entire AI industry. Without it, every new AI application would require training a model from scratch at enormous cost. Because knowledge transfers, a model pre-trained on general text can be fine-tuned for legal analysis, medical coding, or customer support with just thousands of domain-specific examples. This is why foundation models are valuable: they represent a massive upfront investment in general knowledge that can be transferred to any domain. For practitioners, understanding transfer learning helps you predict which tasks will benefit from fine-tuning and which will not. Tasks that are similar to the pre-training distribution transfer well; tasks that are very different may require more data.

In practice

The transfer learning paradigm became dominant with BERT (2018), which showed that pre-training on general text and fine-tuning on specific tasks outperformed task-specific models across the board. Today, every fine-tuned model, from BloombergGPT (finance) to Med-PaLM (medicine) to StarCoder (code), relies on transfer learning. The practical benefit: fine-tuning a foundation model on 10,000 medical examples produces a better medical AI than training a model from scratch on millions of medical records, because the foundation model transfers language understanding, reasoning, and world knowledge. LoRA and other parameter-efficient methods make transfer learning accessible to any team with a few hundred examples.