Vector databases on VKS: pgvector, Milvus, Weaviate

Every RAG project starts with the same shopping list: “we need a vector database.” Three names dominate the shortlist on Kubernetes — pgvector (Postgres extension), Milvus, and Weaviate. They look interchangeable from the marketing pages and behave very differently the moment you push them to a few hundred million embeddings.

This article compares them honestly on VKS: index quality and recall trade-offs, the ops surface (backups, scaling, upgrades), and which one actually fits the maturity of your team. Spoiler — for many internal RAG cases, the answer is not what the AI Twitter consensus says.

TL;DR

• pgvector is “good enough” until your corpus passes a few tens of millions of vectors — then index build cost becomes painful.
• Milvus scales but introduces a control plane your SRE team must learn to love.
• Weaviate trades raw speed for a friendlier ops surface and a built-in modules ecosystem.

What “a vector database” actually has to do

Strip the branding away and the job description is short. That shortness is exactly why three products doing the same four things end up so different to operate.

Approximate nearest neighbour search. An exact k-NN query is a linear scan: compute the distance to every vector, keep the top k. Correct, trivially parallel, and hopeless past a few hundred thousand rows. An ANN index trades a little recall for far fewer distance computations. Two families dominate. Graph indexes — HNSW is the one you will meet everywhere — build a navigable small-world graph over the vectors and answer a query with a greedy walk; recall is tuned at query time (ef_search) and build quality at write time (m, ef_construction). Inverted-file indexes — IVF, usually paired with product or scalar quantization — partition the space into cells and probe only a handful of them; they build faster, use far less memory, and generally need more tuning to reach the same recall.

The recall / latency / build-cost triangle. You get to pick two. Raise ef_search and recall improves while p99 latency climbs. Raise m and ef_construction and both recall and latency improve, but the build takes longer and the write path gets heavier. Quantize and you win memory and build time at the cost of recall. HNSW became the default because it is forgiving on the first two axes and expensive on the third — which is fine until the day you have to rebuild the whole index.

Filtering alongside vector search. This is where RAG systems quietly fail in production, and almost no public benchmark measures it. Real queries are never “nearest ten vectors”; they are “nearest ten vectors this user is allowed to read, from documents not superseded, in the right language”. Post-filtering asks the index for k results and then drops the ones that fail the predicate — you asked for ten, you got three. Pre-filtering restricts the search to the permitted subset, which a graph walk handles badly when selectivity is harsh: the graph’s edges lead mostly to excluded nodes and the traversal degenerates. Engines paper over this differently — bitmap-based filtered traversal, or a fallback to brute force below a cardinality threshold. Test your worst-case tenant, not the average one.

Updates and deletes. Graph indexes are append-friendly and delete-hostile. A delete is normally a tombstone: the vector stays in the graph, is filtered out of results, and keeps costing you memory and traversal steps until a compaction or rebuild reclaims it. A corpus with heavy churn therefore degrades on a schedule nobody planned. And the biggest rebuild trigger is not data at all — it is changing the embedding model. New model, new vector space, every stored vector invalid. Treat a full re-embed as a routine operation you will run several times a year.

pgvector, Milvus, Weaviate side by side on VKS

The interesting differences are not in the ANN math — all three ship HNSW and all three will answer your top-k in single-digit or low double-digit milliseconds on a warm working set. The differences are in shape: how many moving parts land in your cluster, and who gets paged when one of them stops.

pgvector inherits everything you already run. It is an extension, not a product. On VKS you deploy Postgres with an operator you probably already have an opinion about — CloudNativePG, Crunchy, Zalando — and you get a primary plus replicas as a StatefulSet with PVCs on your vSAN storage class. Then you CREATE EXTENSION vector and the vector column is just a column. Every tool in your runbook still applies: PITR from WAL archiving, pg_stat_statements, connection pooling, your existing monitoring, your existing backup schedule, your existing on-call rotation. And filtering is not a feature you have to hope the vendor implemented well — it is SQL, with real joins against real tables:

SELECT c.id, c.doc_id, c.text
FROM chunks c
JOIN doc_acl a ON a.doc_id = c.doc_id
WHERE a.group_id = ANY($2)
  AND c.superseded_at IS NULL
ORDER BY c.embedding <=> $1
LIMIT 10;

The planner decides whether to use the HNSW index or to filter first and scan — and you can inspect that decision with EXPLAIN ANALYZE, which is more than most vector engines let you do. The limits are equally honest: index build is memory-hungry and largely bound by maintenance_work_mem, the index wants to live in page cache, and there is no native sharding of the vector index across nodes. You scale up, then you scale out by partitioning by tenant or collection at the application level.

Milvus brings its own control plane. It is a genuinely distributed system, and it is honest about it: proxies at the front, a set of coordinators, and separate pools of query, data and index nodes. Underneath sit three external dependencies — etcd for metadata, S3-compatible object storage for segments, and a log broker for the write path. That means before you store your first vector you have added three stateful systems to the platform, each with its own upgrade cadence, its own failure modes, and its own dashboards. What you buy for that price is real: index build offloaded to dedicated nodes, storage decoupled from compute, query nodes scaled independently, and support for very large collections that no single machine could hold. The Helm chart and operator make the install easy, which is the trap — installing Milvus is a morning, operating it is a discipline.

Weaviate sits in between. A single Go binary, clustered, with shards holding their own HNSW index and LSM-backed object store, and consensus for schema and membership rather than an external etcd. No broker, no separate object store required for the hot path. You run a StatefulSet, you scale replicas, you get horizontal sharding without adopting a distributed-systems zoo. In exchange you accept an ecosystem that is smaller than Postgres by several orders of magnitude, module behaviour that changes between minor versions, and resharding characteristics you should verify on your own data before you rely on them.

	pgvector	Milvus	Weaviate
Deployment unit	Postgres StatefulSet via an operator	Several component pools + operator	Clustered StatefulSet, one binary
External dependencies	none beyond Postgres	etcd, object storage, log broker	none required
Filtering model	full SQL, joins, planner-visible	native filtered search, engine-decided	native filtered search, engine-decided
Scale-out axis	vertical, then app-level partitioning	horizontal, per component role	horizontal shards per class
Backup path	the one you already run for Postgres	multi-system consistent snapshot	native backup API to object storage
Skills required	Postgres DBA skills you likely have	distributed-systems on-call	Kubernetes plus vendor specifics

Backups, scaling, upgrades: the boring part

Nobody chooses a datastore on its restore procedure, and everybody regrets it. This section is where the decision is actually made.

Backups. With pgvector there is no new backup story, and that is the whole point: base backup plus WAL archiving to object storage gives you point-in-time recovery, and the restore drill your DBA already rehearses covers the vector data for free. Milvus needs a consistent picture across etcd metadata, object-storage segments and in-flight writes; a naive PVC snapshot of the coordinators is not a backup. Weaviate exposes a backup API that pushes to object storage per class, which is clean, but restore semantics and cross-version restore support are exactly the details to verify in your target version rather than assume.

Scaling. Be precise about which resource actually runs out. For ANN work it is almost always memory, not CPU or IOPS: an index that fits in RAM is fast and an index that spills to disk falls off a cliff. pgvector scales by giving the node more RAM and adding read replicas for query fan-out; beyond that you partition by tenant. Milvus scales by adding query nodes, but a loaded collection is resident in query-node memory, so “scale out” means “buy enough aggregate RAM” — the cluster just lets you spread it. Weaviate scales by adding shards, with the caveat that changing the shard count of an existing class is the operation you want to have tested before you need it.

Upgrades. A Postgres minor upgrade is a rolling restart your operator handles; a major upgrade is a planned event with pg_upgrade or logical replication, and you must read the pgvector release notes for index format changes that require a REINDEX — rebuilding a large HNSW index is hours, not minutes, so schedule it like a migration. Milvus upgrades are multi-component and ordered, and you additionally own the upgrade cycles of etcd and the broker underneath. Weaviate is a rolling StatefulSet upgrade, with index-format migrations called out in release notes.

Platform-level details that bite on VKS. Set memory requests equal to limits for any node holding an index, so the kubelet cannot evict a query pod mid-rebuild. Add a PodDisruptionBudget, or a node drain during a VKS lifecycle upgrade will take your quorum with it. Spread replicas across ESXi hosts with anti-affinity. And export the metrics that matter — index build duration, recall on a fixed evaluation set, p99 search latency, and resident index size — into your Prometheus and Grafana stack from the first day, because you cannot argue for a migration without a measured baseline.

Which one to start with

Start with pgvector. This is the unfashionable answer and it is right most of the time.

The argument is arithmetic before it is ideology. A typical internal corpus — an intranet, a product documentation set, a few years of support tickets — lands somewhere between one and ten million chunks once you have split it sensibly:

one chunk               ≈ 300–500 tokens of source text
one vector              = 1024 dims × 4 bytes           ≈ 4 KiB
1 M vectors             ≈ 4 GiB of raw float32
HNSW neighbour lists    ≈ 2 × m × 4 bytes (m = 16)      ≈ 128 B per vector
5 M vectors             ≈ 20 GiB resident, index included

At that size the honest observation is not that one engine beats another on queries per second. It is that the entire working set fits in the RAM of a single reasonably specified node, with room to spare — and a distributed system whose data fits on one machine is a distributed system you are paying for and not using. Halving the storage is also easier than changing engines: cutting the embedding dimension, or storing at half precision where your engine supports it, moves the memory number more than any index tuning will.

Meanwhile the things that actually determine whether your RAG system is good live entirely outside the vector store: chunking strategy, what metadata you attach, whether you re-rank, and whether you have an evaluation set at all. Every hour spent operating a control plane you do not need is an hour not spent on the parts users can feel — a point worth keeping in view when you get to taking RAG to production.

Move when a measured limit forces you, and name the limit. Reasonable triggers, in rough order of how often I see them:

Note what is not on that list: “a benchmark on someone else’s dataset”. Public ANN benchmarks are run on clean, static, unfiltered corpora, which is the one workload you do not have.

If the trigger does fire, the migration is only painful if you skipped the earlier advice. With the chunk text, metadata and ACLs living in a store you control, moving to Milvus or Weaviate is a batch job that reads the source of truth and writes into a new index, run alongside the old one until a comparison on your evaluation set says the new one is at least as good. That is a week of work, not a quarter — and knowing it costs a week is precisely what makes it safe to start simple. The same reasoning applies one layer down, to the Private AI platform architecture itself: adopt the layers you can operate, defer the rest.

Conclusion

Three engines, three operational personalities, and one decision that matters more than the choice between them: whether your vector index is a precious database or a disposable derived artifact. Make it disposable and everything else becomes reversible.