Chain-of-Thought (CoT) Prompting

Chain-of-Thought prompting elicits step-by-step reasoning from an LLM before it produces a final answer. By showing or requesting intermediate reasoning steps, CoT dramatically improves accuracy on arithmetic, logical reasoning, and multi-step problem-solving tasks.

What is Chain-of-Thought Prompting?

Chain-of-Thought (CoT) prompting is a technique that encourages LLMs to produce intermediate reasoning steps before arriving at a final answer. Wei et al. (2022) demonstrated that this simple approach dramatically improved reasoning accuracy on benchmarks like GSM8K and MATH.

Few-Shot CoT

Provide examples that include reasoning steps:

Q: Roger has 5 tennis balls. He buys 2 more cans of 3 balls each. How many tennis balls does he have?
A: Roger starts with 5 balls. 2 cans × 3 balls = 6 balls. 5 + 6 = 11 balls. The answer is 11.

Q: The cafeteria had 23 apples. They used 20 for lunch and bought 6 more. How many apples do they have?
A: [model reasons step by step]

Zero-Shot CoT

Simply append "Let's think step by step" to any question — the model then reasons before answering. This was shown to improve GSM8K accuracy from ~18% to ~79% for certain models.

When CoT Helps Most

  • Arithmetic reasoning — Multi-step calculations.
  • Logical deduction — If-then chains, syllogisms.
  • Planning tasks — Scheduling, resource allocation.
  • Code debugging — Tracing execution flow.

CoT provides little benefit for simple tasks (factual lookup, sentiment classification) and adds unnecessary tokens.

Variants

  • Tree-of-Thought (ToT) — Explores multiple reasoning branches simultaneously.
  • Self-Consistency — Sample multiple reasoning chains, take a majority vote.
  • Program-of-Thoughts — Generate code as the reasoning step, execute it for the answer.

Example

Without CoT: 'What is 17% tip on a $47.50 bill?' → '$8.07'. With CoT: 'First, 10% of $47.50 = $4.75. Then 7% = $3.33. Together = $8.08.' — the reasoning step catches rounding errors and improves reliability.

Related terms

  • Few-Shot Prompting Few-shot prompting is a technique where you include a small number of example input-output pairs in your prompt to guide an LLM's response format and style. It's one of the most effective prompt engineering techniques for improving output quality without fine-tuning.
  • Zero-Shot Prompting Zero-shot prompting asks an LLM to perform a task without providing any examples, relying entirely on the model's pre-trained knowledge and the clarity of the instruction. It works well for common tasks but often requires examples for specialised or nuanced outputs.
  • LLM (Large Language Model) A Large Language Model is a neural network trained on massive text corpora to predict and generate text. LLMs like GPT-4, Claude, and Gemini exhibit broad language understanding, reasoning, and generation capabilities that emerge from scale.
  • In-Context Learning In-context learning is the ability of LLMs to learn new tasks from examples provided directly in the prompt, without updating model weights. By showing the model input-output pairs (few-shot examples), it generalises to new inputs in the same format.