# LLM (Large Language Model)

Canonical: https://aimastermindlab.com/glossary/llm

---
A Large Language Model is a neural network trained on massive text corpora to predict and generate text. LLMs like GPT-4, Claude, and Gemini exhibit broad language understanding, reasoning, and generation capabilities that emerge from scale.

## What is a Large Language Model?

A Large Language Model (LLM) is a neural network — typically a Transformer — trained on hundreds of billions to trillions of words of text to learn the statistical patterns of language. At sufficient scale, LLMs develop emergent capabilities like reasoning, instruction following, code generation, and multilingual understanding.

## Scale and Emergence

The defining characteristic of LLMs is **scale** — both in parameters (billions to trillions) and training data. Below certain sizes, models exhibit limited capabilities. Above certain thresholds, new capabilities emerge that were not present in smaller models. This non-linear relationship between scale and capability was formalized in the **scaling laws** research from OpenAI and DeepMind.

## How LLMs Generate Text

LLMs are *next-token predictors*. Given a sequence of tokens, the model assigns a probability distribution over its entire vocabulary to predict the most likely next token. Repeating this process autoregressively generates fluent, coherent text.

## Modern LLM Families

| Family | Creator | Notable Models |
|---|---|---|
| GPT | OpenAI | GPT-4o, o1, o3 |
| Claude | Anthropic | Claude 3.5 Sonnet, Claude 3 Opus |
| Gemini | Google DeepMind | Gemini 1.5 Pro, Gemini Flash |
| LLaMA | Meta AI | LLaMA 3 8B, 70B, 405B |
| Mistral | Mistral AI | Mistral Large, Mixtral 8x22B |
| Qwen | Alibaba | Qwen2.5-72B |

## Closed vs Open Source

Closed LLMs (GPT-4, Claude) are accessible only via API. Open-weight models (LLaMA, Mistral) can be downloaded and run locally, offering privacy, cost savings, and full customisation at the cost of infrastructure management.

## Example

When you ask ChatGPT to write a Python script, debug your code, or summarise a document, you're interacting with a large language model that generates each word token-by-token based on what is statistically most likely given your prompt.
---

Source: https://aimastermindlab.com/glossary/llm — AI Mastermind Lab (https://aimastermindlab.com)