In-context learning

Definition

A model's ability to learn new tasks from examples provided in the prompt, without any weight updates. In-context learning is what makes few-shot and zero-shot prompting work and is a defining feature of large language models.

Why it matters

In-context learning is the capability that makes LLMs economically revolutionary. Instead of training a separate model for each task (the pre-LLM paradigm), you can give a single model examples in the prompt and it adapts instantly. This collapses the cost of building a new AI feature from months of ML engineering to minutes of prompt design. The deeper implication is that LLMs are not just language models; they are general-purpose pattern matchers that can learn new behaviors at inference time. This is why context window size and context engineering matter so much: they directly determine how many examples and how much context you can provide for in-context learning.

In practice

Google's PaLM paper demonstrated that in-context learning ability scales with model size: 540B-parameter models showed in-context learning on tasks that 8B-parameter models could not. This motivated the push to larger models. In production, companies use in-context learning for rapid prototyping, A/B testing different prompt strategies, and handling long-tail use cases that do not justify fine-tuning. A common pattern: use in-context learning (few-shot) to validate a use case, then graduate to fine-tuning only for the highest-volume tasks where the per-token cost of including examples becomes significant.