# Fine-tuning

Canonical: https://aimastermindlab.com/glossary/fine-tuning

---
Fine-tuning is the process of continuing to train a pre-trained model on a smaller, task-specific dataset to specialise its behaviour. It updates the model's weights, making it more accurate for a specific domain, style, or task without training from scratch.

## What is Fine-tuning?

Fine-tuning takes a pre-trained foundation model and continues training it on a curated, task-specific dataset. The process updates the model's weights to specialise its knowledge or style for a particular use case.

## Types of Fine-tuning

- **Full fine-tuning** — All model weights are updated. Expensive but maximally effective.
- **LoRA (Low-Rank Adaptation)** — Only a small set of adapter matrices are trained. Far more efficient; the dominant approach in 2025-26.
- **QLoRA** — LoRA applied to a quantized base model. Runs on consumer hardware.
- **RLHF (Reinforcement Learning from Human Feedback)** — Fine-tuning with human preference data; how ChatGPT was aligned.
- **DPO (Direct Preference Optimisation)** — A simpler, more stable alternative to RLHF.

## When to Fine-tune vs Use RAG

Fine-tune when you need to change the model's *style, format, or reasoning pattern*. Use RAG when you need to give the model access to *updated or proprietary knowledge*. For most production applications, RAG is faster and cheaper; fine-tuning is reserved for when prompt engineering and RAG are insufficient.

## Data Requirements

High-quality fine-tuning datasets typically need 100–10,000 examples. More data is usually better, but quality matters far more than quantity. Noisy labels actively harm the resulting model.

## Example

OpenAI fine-tuned GPT-3 on human-written conversations to create InstructGPT, which then became the foundation for ChatGPT. The fine-tuning taught the model to follow instructions and decline harmful requests.
---

Source: https://aimastermindlab.com/glossary/fine-tuning — AI Mastermind Lab (https://aimastermindlab.com)