# Vector Databases and Embeddings: From RAG to Ambient AI

> Embeddings map text to vectors for similarity search. But for ambient AI (like Goose Perception), local LLM refiners and SQLite are replacing pure vector DBs.

Published: 2026-03-01  
Canonical: https://aimastermindlab.com/knowledge-base/vector-databases-embeddings

---
## The Traditional Pipeline: Embed, Store, Retrieve

Embeddings are the backbone of modern AI, transforming unstructured inputs into high-dimensional vectors where mathematical distance represents semantic similarity. The classic RAG pipeline relies heavily on this:

- **Models**: Using `text-embedding-3-small` or local `sentence-transformers`.
- **Chunking**: Splitting documents into 256–512 token chunks.
- **Indexing**: Vector DBs use Hierarchical Navigable Small World (HNSW) for fast approximate nearest-neighbor search.
- **Hybrid Search**: Fusing vector embeddings with BM25 (keyword search) to handle exact names, then applying a cross-encoder to re-rank results.

## The Shift to the Edge

As AI moves to local, on-device execution, running heavy distributed vector databases (like Pinecone) isn't always viable. Developers are turning to embedded solutions:
- **ChromaDB**: Runs locally with Python.
- **pgvector**: Adds vector indexing to Postgres.
- **sqlite-vec / sqlite-vss**: The ultimate edge solution, turning standard SQLite into a lightweight vector store.

## Beyond Embeddings: The Ambient AI Approach

Is vector search always the answer? Projects like **Goose Perception** (an open-source ambient macOS assistant) point to a fascinating architectural alternative. When dealing with continuous multi-modal data streams (like OCRing your screen every 20 seconds or transcribing real-time voice), dumping raw embeddings into a vector DB can lead to retrieving noisy, fragmented chunks.

Instead of a traditional Vector DB, the "Ambient AI" architecture uses an LLM as a Refiner:
1. **Raw Storage**: Capture raw text (from Vision OCR or Whisper) into a standard relational database (like SQLite).
2. **LLM Refiners**: Instead of embedding raw text, use a local, quantized LLM (e.g., Qwen2.5-3B via MLX) to periodically scan the raw data and extract structured knowledge—Projects, Collaborators, Interests, and TODOs.
3. **Knowledge Graph Over Vectors**: The LLM writes this refined data back into relational tables.

**Why this matters**: This approach builds a deterministic, queryable knowledge graph instead of relying on probabilistic vector similarity. For an ambient agent watching your screen, knowing that "Project X" reliably involves "Person Y" is often better solved by an LLM Refiner writing to SQLite than by embedding thousands of scattered screen captures.

## Conclusion

Vector databases remain essential for querying massive corpuses. But for personal, local AI, the paradigm is shifting. The most original designs blend lightweight local vector search with continuous LLM refinement, ensuring agents actually understand the entities within data rather than just finding similar text.
---

Source: https://aimastermindlab.com/knowledge-base/vector-databases-embeddings — AI Mastermind Lab (https://aimastermindlab.com)