Vector Databases Compared: Pinecone vs Weaviate vs Qdrant vs pgvector

HNSW index mechanics, the metadata filtering problem (pre-filter vs post-filter), hybrid search support, and the decision framework for choosing a vector database. When pgvector is the right answer.

Vector Databases Are Not the Same Product

Pinecone, Weaviate, Qdrant, pgvector, Milvus, Chroma — these are all 'vector databases' but they make fundamentally different tradeoffs. Choosing the wrong one for your use case costs you months of migration work. The decision comes down to: managed vs. self-hosted, pure vector vs. hybrid, scale, and whether your retrieval is standalone or needs to join relational data.

What Every Vector Database Actually Does

Every vector database does the same core thing: store high-dimensional vectors, build an ANN index (HNSW, IVF, or similar), and answer nearest-neighbor queries. The differences are in everything else — filtering, multi-tenancy, hybrid search, consistency guarantees, operational overhead, and cost.

HNSW: The Index That Runs Most Production Systems

Hierarchical Navigable Small World graphs are the dominant ANN index in production because they offer excellent recall-latency tradeoffs with no training step. Unlike IVF indexes which require a k-means clustering pass before indexing, HNSW builds incrementally. This matters for production: you can add new vectors without rebuilding the entire index.

HNSW has two critical parameters: M (number of connections per node, controls recall and memory) and efConstruction (search depth during construction, controls index quality at build time). Higher M → better recall, more memory. For most production use cases: M=16 and efConstruction=128 are sensible defaults. Tuning these without benchmarking on your data is guessing.

The Metadata Filtering Problem

Real queries are rarely pure semantic search. You want 'find similar products to X, but only in category Electronics, and only available in the user's country.' Filtering on metadata while doing ANN search is surprisingly hard. ANN indexes are built globally — filtering after ANN retrieval loses recall (relevant items outside the top-K are excluded). Filtering before ANN search (pre-filtering) requires per-filter index segmentation.

Different vector DBs handle this differently. Qdrant uses a quantized payload index for efficient pre-filtering. Weaviate uses inverted indexes for metadata combined with vector search. Pinecone's metadata filtering is a post-filter by default (recall degrades with selective filters). pgvector gets regular Postgres WHERE clauses — familiar but slow on large datasets.

The Comparison

Hybrid Search: When Semantic Isn't Enough

Semantic search (vector-only) fails on exact lookups — product IDs, person names, rare technical terms, specific codes. Keyword search (BM25/TF-IDF) fails on paraphrase and conceptual queries. Production search almost always needs both, combined with Reciprocal Rank Fusion or a learned linear combination.

Weaviate and Qdrant have first-class hybrid search. Pinecone supports it via sparse+dense vector pairs. If your use case has any exact-match requirements (and most enterprise search does), evaluate hybrid search quality before choosing a vector database — not every system's hybrid implementation performs equally.

When NOT to Use a Dedicated Vector Database

Under 1 million vectors: pgvector with an HNSW index is fast, free, and keeps everything in one system. You don't need Pinecone for 50,000 documents. Under 100,000 vectors: FAISS in memory is often the right answer — no infrastructure, full control. The 'we need a vector database' impulse often precedes the 'we need to build something that actually works' step.