# In-Context Learning

Canonical: https://aimastermindlab.com/glossary/in-context-learning

---
In-context learning is the ability of LLMs to learn new tasks from examples provided directly in the prompt, without updating model weights. By showing the model input-output pairs (few-shot examples), it generalises to new inputs in the same format.

## What is In-Context Learning?

In-context learning (ICL) is a striking emergent property of large language models: they can perform new tasks simply by seeing a few examples in the prompt, without any gradient updates to their weights. This is fundamentally different from machine learning, where training always involves weight updates.

## Zero-shot, One-shot, Few-shot

- **Zero-shot** — No examples; the model relies on instructions alone: "Classify the sentiment of this review: [review]"
- **One-shot** — One example provided before the actual query.
- **Few-shot** — Typically 3-10 carefully chosen examples demonstrating the desired input-output format.

## Why Few-shot Works

Research suggests ICL exploits the model's training data distribution — the model essentially pattern-matches the prompt structure to similar patterns from its training corpus, rather than truly "learning" in an algorithmic sense. Few-shot examples help the model identify the *task* and *output format*, not just teach it new knowledge.

## Best Practices for Few-shot Prompting

1. **Use diverse examples** — Cover edge cases and different input types.
2. **Order matters** — The most recent example (closest to the actual query) has the strongest influence.
3. **Consistent formatting** — The model is sensitive to formatting inconsistencies.
4. **Label quality** — Wrong examples make the model worse than no examples.

## Limitations

ICL is limited by context window size — you can only provide as many examples as fit. For tasks requiring hundreds of examples, supervised fine-tuning is more reliable.

## Example

Giving GPT-4 three examples of (French sentence → English translation) in the prompt, then asking it to translate a new sentence, is one-shot/few-shot in-context learning — no fine-tuning required.
---

Source: https://aimastermindlab.com/glossary/in-context-learning — AI Mastermind Lab (https://aimastermindlab.com)