# Hybrid Search: BM25 + Vector Similarity (With Code)

> Why relying solely on vector search for RAG is a mistake. Learn how to implement Hybrid Search combining lexical (BM25) and semantic (Vector) search using LangChain and multiple retrievers.

Published: 2026-03-08  
Canonical: https://aimastermindlab.com/tutorials/hybrid-search-bm25-vector-similarity

---
> **TL;DR:** Pure vector search is amazing at understanding *concepts*, but terrible at finding exact matches (like serial numbers, specific names, or error codes). **Hybrid search** solves this by querying both a vector database and a traditional keyword index (BM25), then intelligently merging the results.

If you've built a basic RAG application, you've likely encountered this problem:

**User query:** *"How do I fix Error Code XJ-881?"*

A pure vector database will embed that query and look for things *conceptually similar* to fixing errors. It might return completely irrelevant documents about "Troubleshooting network issues" because "XJ-881" didn't carry enough semantic weight in the embedding model.

To build production-grade RAG, you must implement **Hybrid Search**.

## The Two Pillars of Hybrid Search

1. **Semantic Search (Vector Similarity)**: "Understands" meaning. A search for "canines" will find documents talking about "dogs" and "wolves."
   - *Tool*: ChromaDB, Pinecone, FAISS.
2. **Lexical Search (BM25)**: "Understands" keywords. A search for "XJ-881" looks precisely for the frequency of that exact string in your documents and penalizes overly common words (the TF-IDF approach).
   - *Tool*: Elasticsearch, BM25 algorithm.

Hybrid means doing both simultaneously, then fusing the results using an algorithm like Reciprocal Rank Fusion (RRF) or a weighted sum.

## Implementation with LangChain

We can build this easily in Python using LangChain's `EnsembleRetriever`.

### Prerequisites

```bash
pip install langchain langchain-openai langchain-community rank_bm25 chromadb
```

*(Note: `rank_bm25` is necessary for the local lexical search implementation.)*

### Step 1: Set up the Documents

```python
from langchain_core.documents import Document

# Sample dataset showing the weakness of pure vector search
docs = [
    Document(page_content="To fix the network router, reboot the system."),
    Document(page_content="Error Code XJ-881: The thermal paste has degraded."),
    Document(page_content="General overheating issues can cause system failure."),
    Document(page_content="For support regarding part XJ-900, call 555-0199.")
]
```

### Step 2: Create the Vector Retriever (Semantic)

First, we build the standard dense embedding retriever using OpenAI (or [Ollama locally](/tutorials/run-llm-locally-mac-ollama)).

```python
from langchain_openai import OpenAIEmbeddings
from langchain_community.vectorstores import Chroma

embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
vectorstore = Chroma.from_documents(docs, embeddings)

# Retriever 1: Vector Search
vector_retriever = vectorstore.as_retriever(search_kwargs={"k": 2})
```

### Step 3: Create the BM25 Retriever (Lexical)

Next, we use LangChain's built-in BM25 interface to index the exact same documents.

```python
from langchain_community.retrievers import BM25Retriever

# Retriever 2: Keyword Search
bm25_retriever = BM25Retriever.from_documents(docs)
bm25_retriever.k = 2 # Return top 2 results
```

### Step 4: The Ensemble Retriever (Hybrid)

Now we fuse them. The `EnsembleRetriever` runs both retrievers in parallel and merges their results. 

We provide `weights` to determine which is more important. `[0.5, 0.5]` means equal weighting. If your application relies heavily on specific product names, you might lean towards `[0.3, 0.7]` (30% Semantic, 70% Keyword).

```python
from langchain.retrievers import EnsembleRetriever

# Combine them!
hybrid_retriever = EnsembleRetriever(
    retrievers=[vector_retriever, bm25_retriever],
    weights=[0.5, 0.5]
)
```

### Step 5: Test the Difference

Let's search for an exact error code and see why Hybrid is necessary.

```python
query = "I have Error Code XJ-881"

# 1. PURE VECTOR SEARCH
print("=== VECTOR ONLY ===")
vector_results = vector_retriever.invoke(query)
for doc in vector_results:
    print(f"- {doc.page_content}")

# Depending on the embedding model, this often returns the "overheating" 
# or "router" generic IT text because "XJ-881" is ignored mathematically.

# 2. HYBRID SEARCH
print("\n=== HYBRID SEARCH ===")
hybrid_results = hybrid_retriever.invoke(query)
for doc in hybrid_results:
    print(f"- {doc.page_content}")

# The BM25 algorithm immediately catches the "XJ-881" string, pulling the correct
# document to the top of the ensemble rankings.
```

## Re-Ranking: The Next Step

While this `EnsembleRetriever` uses simple weight mathematics, production RAG systems (like those used by Perplexity) take Hybrid search a step further.

They retrieve ~20 documents using Hybrid search, and then pass those 20 documents to a **Cross-Encoder Model** (like `Cohere Rerank` or `bge-reranker`). The cross-encoder is an AI model that explicitly scores exactly how well a specific document answers a specific query, resorting the Top 20 into a highly precise Top 3 to pass to the final LLM.

Implementing Hybrid Search + Re-ranking is the single most effective way to eliminate hallucinations in a RAG pipeline.
---

Source: https://aimastermindlab.com/tutorials/hybrid-search-bm25-vector-similarity — AI Mastermind Lab (https://aimastermindlab.com)