Fine-tuning

Fine-tuning is the process of continuing to train a pre-trained model on a smaller, task-specific dataset to specialise its behaviour. It updates the model's weights, making it more accurate for a specific domain, style, or task without training from scratch.

What is Fine-tuning?

Fine-tuning takes a pre-trained foundation model and continues training it on a curated, task-specific dataset. The process updates the model's weights to specialise its knowledge or style for a particular use case.

Types of Fine-tuning

  • Full fine-tuning — All model weights are updated. Expensive but maximally effective.
  • LoRA (Low-Rank Adaptation) — Only a small set of adapter matrices are trained. Far more efficient; the dominant approach in 2025-26.
  • QLoRA — LoRA applied to a quantized base model. Runs on consumer hardware.
  • RLHF (Reinforcement Learning from Human Feedback) — Fine-tuning with human preference data; how ChatGPT was aligned.
  • DPO (Direct Preference Optimisation) — A simpler, more stable alternative to RLHF.

When to Fine-tune vs Use RAG

Fine-tune when you need to change the model's style, format, or reasoning pattern. Use RAG when you need to give the model access to updated or proprietary knowledge. For most production applications, RAG is faster and cheaper; fine-tuning is reserved for when prompt engineering and RAG are insufficient.

Data Requirements

High-quality fine-tuning datasets typically need 100–10,000 examples. More data is usually better, but quality matters far more than quantity. Noisy labels actively harm the resulting model.

Example

OpenAI fine-tuned GPT-3 on human-written conversations to create InstructGPT, which then became the foundation for ChatGPT. The fine-tuning taught the model to follow instructions and decline harmful requests.

Related terms

  • LoRA (Low-Rank Adaptation) LoRA is a parameter-efficient fine-tuning technique that trains only a small set of low-rank adapter matrices instead of updating all model weights. It reduces the number of trainable parameters by up to 10,000x while achieving fine-tuning quality comparable to full parameter updates.
  • RLHF (Reinforcement Learning from Human Feedback) RLHF is the training technique used to align LLMs with human preferences. Human annotators rank model outputs, those preferences train a reward model, and the LLM is then fine-tuned using reinforcement learning to maximise the reward signal. RLHF is how ChatGPT was trained to be helpful and safe.
  • LLM (Large Language Model) A Large Language Model is a neural network trained on massive text corpora to predict and generate text. LLMs like GPT-4, Claude, and Gemini exhibit broad language understanding, reasoning, and generation capabilities that emerge from scale.
  • RAG (Retrieval-Augmented Generation) RAG is a technique that combines an LLM with an external knowledge retrieval system. Instead of relying solely on its training data, the model retrieves relevant documents at query time and uses them as context to generate more accurate, up-to-date responses.
  • Quantization Quantization reduces the numerical precision of model weights (e.g. from 32-bit floats to 4-bit integers), shrinking memory requirements and accelerating inference with minimal quality loss. It's the key technique for running large models on consumer hardware.