Milvus: A Distributed Vector Database for AI

Understanding Milvus Vector Database

Milvus is an open-source, cloud-native vector database written in Go and C++ that stores embeddings alongside scalar fields like integers, strings, and JSON, then serves nearest-neighbor search over them at scale. It runs as a distributed, Kubernetes-native cluster for production, or as Milvus Lite, a single-file Python library you `pip install` for local prototyping.

The Challenge of Unstructured Data at Scale

AI applications that turn text, images, or other unstructured data into vectors need somewhere to store and query those vectors at volumes a general-purpose database wasn't built for — Milvus's own README frames this as handling tens of thousands of search queries against billions of vectors while keeping data fresh with real-time streaming updates. The specific gap it targets is scaling read-heavy vector search and write-heavy ingestion independently, by separating compute from storage instead of running both on one node.

Core Capabilities of Milvus

• ✓Distributed architecture that separates compute from storage, so you scale query nodes for read-heavy traffic and data nodes for write-heavy ingestion independently — a distributed-systems trade-off, not just a scaling toggle.
• ✓Five index types in one engine — HNSW, IVF, SCANN, DiskANN, and FLAT brute-force, each featuring quantization-based variants — and mmap, so ANN search behavior is tunable per collection.
• ✓Metadata filtering and range search alongside vector similarity, for queries that need to narrow results by scalar fields (price, date, category) before or during the nearest-neighbor pass.
• ✓GPU indexing, including NVIDIA's CAGRA, on top of CPU hardware acceleration.
• ✓Native hybrid search: dense vector similarity plus full-text search via BM25 and learned sparse embeddings like SPLADE and BGE-M3, storable in the same collection with result reranking.
• ✓Multi-tenancy isolation at the database, collection, partition, or partition-key level — the README says a single cluster can handle hundreds to millions of tenants.
• ✓Hot/cold storage tiering: frequently accessed data stays in memory or on SSD, less-accessed data moves to cheaper storage to cut cost.
• ✓Mandatory authentication, TLS encryption, and role-based access control (RBAC).

Real-World Applications Powered by Milvus

• •Retrieval-Augmented Generation (RAG) pipelines that need a vector store behind an LLM to ground responses in retrieved documents.
• •Text and image similarity search, using dense embeddings to find near-duplicates or semantically related content.
• •Recommendation systems built on embedding similarity between users, items, or content.
• •Hybrid search products that combine semantic (vector) ranking with BM25 keyword matching in a single query.

Getting Started with Milvus

For local prototyping, `pip install -U pymilvus` gets you the Python SDK, and adding `pip install pymilvus[milvus-lite]` gives you Milvus Lite so `MilvusClient("milvus_demo.db")` runs a local vector database backed by a single file — no server needed. For a real deployment, the README points to Standalone mode (single-machine, documented via Docker) or a fully-distributed, Kubernetes-native cluster; you can also skip self-hosting with Zilliz Cloud's Serverless, Dedicated, or BYOC options. Building from source needs Go >= 1.21, CMake >= 3.26.4 and < 4, GCC >= 11 (Linux) or llvm >= 15 (macOS), and Python > 3.8 and <= 3.11 — then `git clone`, `./scripts/install_deps.sh`, and `make`.

Interacting with Milvus: Code Examples

Connect with `from pymilvus import MilvusClient`, then point it at a local file for Milvus Lite, or at a running server or Zilliz Cloud endpoint using a `uri` and `token`. Create a collection with a fixed vector dimension — the README's example is `client.create_collection(collection_name="demo_collection", dimension=768)` for 768-dimension embeddings — then load data with `client.insert(collection_name="demo_collection", data=data)`. Querying is `client.search(...)` with a list of query vectors, a `limit` for how many results (topK) to return, and `output_fields` to pick which stored fields come back.

Exploring Other Vector Database Options

Common Questions About Milvus

Who should try it — and who should skip

Reach for Milvus if you're building a RAG or search feature that needs to scale past what a single-node database can handle, and your team already runs Kubernetes — the compute/storage separation and replica model are built for that operational reality. Skip it if you've got a few hundred thousand to a few million vectors and no K8s experience: Milvus Lite covers local prototyping fine, but running the real distributed system as your production backend to search a small embedding set means taking on cluster operations you probably don't need yet — pgvector or a managed service gets there with less overhead.

Related repositories