Chroma is an open-source embedding database aimed at LLM applications. It stores documents, their embeddings, and metadata in named collections and answers similarity queries with optional metadata filters. The examples above show the basic add and query calls; the sections below explain the architecture, a fuller persistent example, and how Chroma compares with other vector stores.
Here's a Python example demonstrating how to add records (documents and embeddings) to a Chroma vector database:
Make sure you have ChromaDB installed:
pip install chromadb
import chromadb
from chromadb.utils import embedding_functions
# Initialize a Chroma client (in-memory) - you can change this to persistent storage
client = chromadb.Client()
# Create a collection (similar to a table in a relational database)
collection_name = "my_collection"
try:
collection = client.create_collection(name=collection_name)
except:
collection = client.get_collection(name=collection_name) # If already exists, get it
# Example documents and their embeddings (replace with your actual data)
documents = [
"This is the first document.",
"This is the second document about cars.",
"This is the third document discussing programming.",
"This is the fourth document about artificial intelligence."
]
# Generate embeddings using a sentence transformer model
sentence_transformer_ef = embedding_functions.SentenceTransformerEmbeddingFunction(model_name="all-MiniLM-L6-v2")
collection.update_embeddings_function(sentence_transformer_ef)
# Add the documents to the collection (along with some metadata - optional)
ids = ["doc1", "doc2", "doc3", "doc4"]
metadatas = [
{"source": "website1", "author": "userA"},
{"source": "website2", "author": "userB"},
{"source": "blog", "author": "userC"},
{"source": "article", "author": "userD"}
]
collection.add(
documents=documents,
ids=ids,
metadatas=metadatas,
)
print("Records added to Chroma collection.")
Now, let's see how to retrieve records from the Chroma collection:
import chromadb
from chromadb.utils import embedding_functions
# Initialize a Chroma client (in-memory)
client = chromadb.Client()
# Get the collection
collection_name = "my_collection"
collection = client.get_collection(name=collection_name)
# Example query
query = "Tell me about cars"
# Generate the embedding for the query
query_embedding = sentence_transformer_ef.encode(query)
# Perform a similarity search
results = collection.query(
query_embeddings=[query_embedding],
n_results=2 # Retrieve the top 2 most similar records
)
# Print the results
print("Query Results:")
print(results)
# Access the documents
print("\nDocuments:")
print(results['documents'][0])
# Access the metadatas
print("\nMetadatas:")
print(results['metadatas'][0])
# Access the ids
print("\nIds:")
print(results['ids'][0])
Remember to replace the example data and query with your own data and queries.
A Chroma client talks to collections, which play the role of tables. Each record has a string id, an optional document, an embedding, and a flat metadata dictionary. If you pass documents without embeddings, the collection's embedding function computes them; the default uses a small sentence-transformer model run locally. Vectors are indexed with HNSW for approximate nearest-neighbour search, while documents and metadata are kept in an embedded SQLite database. Chroma can run in-process (ephemeral or persistent to a directory) or as a server that clients reach over HTTP.
The design favours simplicity over distributed scale. In embedded mode there is no separate service to deploy, and the whole database is a directory you can copy, version, or delete. That makes Chroma a common default in LangChain and LlamaIndex tutorials and in notebooks where a developer wants retrieval working in a few lines. In client/server mode the same API is exposed over HTTP, so an application can move from local development to a shared server without changing its query code.
collection.query embeds the query text with the same embedding function used at insert time, applies any where metadata filter and where_document content filter, and returns the nearest records with their distances. The distance metric is chosen when the collection is created through its HNSW configuration, and cannot be changed afterwards without rebuilding the collection.
import chromadb
from chromadb.utils import embedding_functions
client = chromadb.PersistentClient(path="./chroma_db")
ef = embedding_functions.SentenceTransformerEmbeddingFunction(model_name="all-MiniLM-L6-v2")
docs = client.get_or_create_collection(
name="kb", embedding_function=ef, metadata={"hnsw:space": "cosine"})
docs.upsert(
ids=["a1", "a2", "a3"],
documents=["Reset a password from the login page.",
"Invoices are emailed on the first of the month.",
"Two-factor auth uses an authenticator app."],
metadatas=[{"topic": "account"}, {"topic": "billing"}, {"topic": "account"}],
)
res = docs.query(query_texts=["how do I change my password"], n_results=2,
where={"topic": "account"})
for i, d, dist in zip(res["ids"][0], res["documents"][0], res["distances"][0]):
print(i, round(dist, 3), d)
docs.delete(where={"topic": "billing"})
print(docs.count())upsert makes ingestion idempotent: rerunning a loader updates records with the same ids instead of failing on duplicates.
Two practical notes on ingestion. First, batch your inserts: passing hundreds or thousands of records per call is much faster than adding one at a time, because embedding and index updates are amortized. Second, keep ids stable and meaningful, for example a document path plus a chunk number, so that re-ingesting a changed file can delete its old chunks by a metadata filter and upsert the new ones without leaving orphans behind.
| Option | Best for | Consider instead when |
|---|---|---|
| Chroma | Prototypes, local RAG, small to medium collections with metadata | You need multi-node scale or strict SLAs |
| FAISS | Maximum speed on large static sets, GPU | You need filters, updates, persistence |
| pgvector | Data already in Postgres, joins with relational data | Very high-dimensional, very large indexes |
| Weaviate | Server deployment, hybrid search, modules | You want a zero-ops embedded library |
chromadb.Client() is in-memory: data disappears when the process exits. Use PersistentClient or a server.create_collection fails if the name exists; use get_or_create_collection.Back up the persistence directory while no writer is active, and pin the chromadb version, since on-disk formats have changed between releases.