ChromaDB Tutorial for Python

Learn ChromaDB in Python with a step-by-step tutorial covering collections, embeddings, metadata filtering, document retrieval, and basic RAG workflows.

By AI Mastermind Lab · Published 8 Mar 2026

TL;DR: ChromaDB is an open-source, lightweight vector database designed for AI applications. It stores document embeddings locally, supports similarity search and metadata filtering, and integrates directly with LangChain and LlamaIndex. Install it with pip install chromadb, create a collection, add documents, and query by semantic similarity — all in under 10 lines of Python.

If you are building a Retrieval-Augmented Generation (RAG) pipeline or any AI application that needs semantic search, ChromaDB is one of the fastest ways to get started. This chromadb tutorial for Python walks you through everything: installation, creating collections, adding documents with metadata, querying by similarity, using persistent storage, and integrating Chroma into a RAG pipeline.

What Is ChromaDB?

ChromaDB is an open-source embedding database (vector database) purpose-built for AI. It stores text or pre-computed embeddings as vectors and retrieves them by semantic similarity — meaning it finds documents that are conceptually related to your query, not just keyword matches.

Key features:

  • Runs locally with zero infrastructure (just pip install)
  • Automatic embedding generation via built-in models
  • Metadata filtering on stored documents
  • Persistent storage to disk
  • LangChain, LlamaIndex, and OpenAI integrations
  • Open source (Apache 2.0)

Installation

pip install chromadb

That is it. No database server, no Docker, no configuration files. ChromaDB runs entirely in-process by default. For the full API reference, see the official ChromaDB documentation.

Creating a Collection

Collections are ChromaDB's equivalent of tables. Each collection stores documents, their embeddings, and optional metadata:

import chromadb

# Create an in-memory client
client = chromadb.Client()

# Create a collection (or get it if it already exists)
collection = client.get_or_create_collection(name='my_documents')
print(f'Collection: {collection.name}, Count: {collection.count()}')

By default, ChromaDB uses its built-in embedding model (a variant of all-MiniLM-L6-v2) to automatically embed your text. You can also provide your own embeddings.

Adding Documents and Metadata

Add documents with unique IDs and optional metadata for filtering:

collection.add(
    documents=[
        'RAG combines retrieval with generation to produce grounded answers.',
        'ChromaDB is a lightweight vector database for AI applications.',
        'FAISS is a library for efficient similarity search by Meta AI.',
        'LangChain provides tools for building LLM-powered applications.',
        'Fine-tuning adapts a pre-trained model to a specific task.',
    ],
    metadatas=[
        {'topic': 'rag', 'difficulty': 'beginner'},
        {'topic': 'vector_db', 'difficulty': 'beginner'},
        {'topic': 'vector_db', 'difficulty': 'intermediate'},
        {'topic': 'framework', 'difficulty': 'beginner'},
        {'topic': 'training', 'difficulty': 'advanced'},
    ],
    ids=['doc1', 'doc2', 'doc3', 'doc4', 'doc5']
)
print(f'Added {collection.count()} documents')

Important: Each document must have a unique id. If you add a document with an existing ID, it will be updated (upserted), not duplicated.

Querying by Semantic Similarity

Query the collection with natural language. ChromaDB embeds your query and finds the most similar documents:

results = collection.query(
    query_texts=['What databases work well for AI search?'],
    n_results=3
)

for i, doc in enumerate(results['documents'][0]):
    distance = results['distances'][0][i]
    metadata = results['metadatas'][0][i]
    print(f'{i+1}. [{distance:.4f}] {doc}')
    print(f'   Metadata: {metadata}')

Lower distance = higher similarity. The results are ranked by cosine distance by default.

Metadata Filtering

ChromaDB supports powerful metadata filters to narrow your search. Combine similarity search with structured filtering:

# Find beginner-level documents about vector databases
results = collection.query(
    query_texts=['vector search for AI'],
    n_results=3,
    where={'topic': 'vector_db'}  # Only search docs with this metadata
)

for doc in results['documents'][0]:
    print(doc)

Supported filter operators include $eq, $ne, $gt, $lt, $in, and $nin. You can combine multiple filters with $and and $or:

# Complex filter: beginner OR intermediate difficulty
results = collection.query(
    query_texts=['database for AI'],
    n_results=5,
    where={
        '$or': [
            {'difficulty': 'beginner'},
            {'difficulty': 'intermediate'}
        ]
    }
)

Using the Persistent Client

By default, ChromaDB stores data in memory (lost when the script exits). For persistent storage across sessions, use PersistentClient:

import chromadb

# Data is saved to and loaded from this directory automatically
client = chromadb.PersistentClient(path='./chroma_db')

collection = client.get_or_create_collection(name='my_documents')
# Data persists across script restarts

This stores the database as SQLite files on disk. No separate server process needed — ChromaDB reads and writes directly to the specified directory.

Using ChromaDB in a RAG Pipeline

The most common use case for ChromaDB is powering the retrieval step in a RAG pipeline. Here is a minimal example with LangChain:

from langchain_community.vectorstores import Chroma
from langchain_openai import OpenAIEmbeddings, ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnablePassthrough
from langchain_core.output_parsers import StrOutputParser

# Create vector store with OpenAI embeddings
embeddings = OpenAIEmbeddings(model='text-embedding-3-small')
vectorstore = Chroma(
    persist_directory='./chroma_db',
    embedding_function=embeddings
)

# Build RAG chain
retriever = vectorstore.as_retriever(search_kwargs={'k': 3})
llm = ChatOpenAI(model='gpt-4o', temperature=0)

prompt = ChatPromptTemplate.from_template("""
Answer based only on the context below.

Context: {context}
Question: {question}
""")

def format_docs(docs):
    return '\n'.join(d.page_content for d in docs)

chain = (
    {'context': retriever | format_docs, 'question': RunnablePassthrough()}
    | prompt | llm | StrOutputParser()
)

answer = chain.invoke('What is ChromaDB used for?')
print(answer)

For the full RAG walkthrough, see our LangChain RAG Tutorial in Python.

When ChromaDB Is a Good Fit

ChromaDB excels for:

  • Prototyping and local development — Zero setup, no server, instant start
  • Small to medium datasets — Up to ~1 million documents on a single machine
  • RAG pipelines — Direct integration with LangChain and LlamaIndex
  • Privacy-sensitive workloads — Everything runs locally

Consider alternatives when:

  • Scale exceeds millions of vectors — Look at Pinecone (managed) or Weaviate (self-hosted) for distributed deployments
  • You need advanced indexing — FAISS offers more index types (IVF, HNSW, PQ) for fine-tuned performance at scale
  • Production multi-tenancy — Managed solutions like Pinecone handle tenant isolation and scaling automatically

For a detailed comparison, see Vector Databases for RAG.

FAQ

Q: Does ChromaDB require a running server? A: No. In default mode, ChromaDB runs entirely in-process — no separate server. For multi-client setups, Chroma also offers an HttpClient that connects to a Chroma server, but most local development uses the embedded client.

Q: What embedding model does ChromaDB use by default? A: ChromaDB uses all-MiniLM-L6-v2 (via the Sentence Transformers library) by default. You can override this with any embedding model — OpenAI, Cohere, Hugging Face, or custom models.

Q: Can ChromaDB handle images or other data types? A: ChromaDB is expanding multimodal support. Currently, it works best with text and pre-computed embeddings. For image search, compute your own image embeddings (e.g., with CLIP) and store them in Chroma.

Q: How does ChromaDB compare to FAISS? A: ChromaDB is a full database with persistence, metadata filtering, and document storage. FAISS is a lower-level library focused purely on vector similarity search — it does not store documents or metadata. ChromaDB is easier to use; FAISS offers more control over indexing algorithms. See our comparison in Vector Databases for RAG.

Q: Can I use ChromaDB with Ollama? A: Yes. Use OllamaEmbeddings from LangChain to embed documents locally, then store them in ChromaDB. Combined with an Ollama-hosted LLM, you get a fully local, zero-cost RAG system. See our Ollama guide for setup.

Related Resources