# Embedding Canonical: https://aimastermindlab.com/glossary/embedding --- An embedding is a numerical vector representation of data (text, image, audio) that captures semantic meaning. Similar concepts have embeddings that are close together in vector space, enabling AI systems to measure similarity, cluster data, and retrieve relevant information. ## What is an Embedding? An embedding is a fixed-length numerical vector that represents a piece of data — most commonly text — in a high-dimensional space. The key property is that **semantically similar items have vectors that are close together**, as measured by cosine similarity or dot product distance. ## How Text Embeddings are Created Text embeddings are produced by neural networks trained on massive corpora. Models like OpenAI's `text-embedding-3-small`, Google's `text-embedding-004`, or open-source `nomic-embed-text` process input text and output a vector of typically 768–3072 float values. ## Why Embeddings Are Fundamental Almost every modern AI application relies on embeddings: - **RAG systems** use embeddings to retrieve semantically relevant documents. - **Recommendation systems** find similar items by comparing embedding distances. - **Classification** models use embeddings as input features. - **Semantic search** replaces keyword search with meaning-based search. ## Embedding Models vs Generation Models Embedding models output vectors, not text. They are separate from generative LLMs. You typically use an embedding model to index your data, and a generative LLM to produce the final response. ## Dimensions and Trade-offs Higher-dimensional embeddings capture more nuance but are slower and more expensive to store. Techniques like **matryoshka representation learning** (used in OpenAI's latest models) allow you to truncate dimensions without significant quality loss. ## Example The sentences 'The cat sat on the mat' and 'A feline rested on a rug' produce embeddings that are very close in vector space, even though they share no exact words. --- Source: https://aimastermindlab.com/glossary/embedding — AI Mastermind Lab (https://aimastermindlab.com)