Your RAG pipeline is returning mediocre results. You have tried tweaking the prompt, adjusting chunk sizes, even swapping embedding models. The retrieval quality barely moves. The problem might not be your LLM or your embedding model. It might be that you are using your vector database as a dumb key-value store when it is capable of being the most sophisticated component in your stack.

Qdrant is an open-source vector search engine written in Rust. It is also one of the most feature-rich retrieval systems available, and most teams use less than 30% of what it can do. This post is the architectural map you need.

The Data Model: More Than Vectors

The fundamental unit in Qdrant is a point — a record that combines a vector, an optional JSON payload, and an ID.

{
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "vector": [0.1, 0.2, 0.3, 0.4],
    "payload": {
        "category": "regulatory",
        "region": "EU",
        "effective_date": "2026-01-15T00:00:00Z",
        "tags": ["GxP", "AI", "validation"]
    }
}

Points live in collections — named sets that define the vector space. Each collection specifies a distance metric (Cosine, Dot Product, Euclidean, or Manhattan) and a vector dimensionality. But here is where it gets interesting: a single point can carry multiple named vectors, each with its own dimension and metric.

This is the foundation for everything that follows.

┌─────────────────────────────────────────────────────────────┐
│                    SINGLE POINT                              │
│                                                              │
│  ┌────────────────────────────────────────────────────────┐  │
│  │  "dense-embedding"     Cosine, 1536-dim                │  │
│  │  [0.012, -0.034, 0.056, ..., 0.078]                   │  │
│  └────────────────────────────────────────────────────────┘  │
│                                                              │
│  ┌────────────────────────────────────────────────────────┐  │
│  │  "bm25-sparse"         IDF-weighted                    │  │
│  │  indices: [112174620, 177304315, 662344706]            │  │
│  │  values:  [1.669,      1.669,      1.669]              │  │
│  └────────────────────────────────────────────────────────┘  │
│                                                              │
│  ┌────────────────────────────────────────────────────────┐  │
│  │  "colbert-tokens"      Multi-vector, 128-dim × N       │  │
│  │  [[0.1, 0.2, ...], [0.3, 0.4, ...], ...]              │  │
│  └────────────────────────────────────────────────────────┘  │
│                                                              │
│  ┌────────────────────────────────────────────────────────┐  │
│  │  PAYLOAD (JSON)                                        │  │
│  │  { "category": "regulatory", "region": "EU" }          │  │
│  └────────────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────────────┘

Three vector types are supported:

Type Structure Use Case
Dense Fixed-length float array Standard neural embeddings (OpenAI, Cohere, sentence-transformers)
Sparse Index/value pairs, variable length BM25 keyword search, SPLADE, collaborative filtering
Multi-vector Fixed-width matrix, variable height ColBERT late-interaction models

Most teams only use dense vectors. Adding sparse vectors for lexical matching is the single highest-impact upgrade most RAG systems can make.

The Payload Is a Database Inside Your Database

Qdrant’s payload system is not a tagging afterthought. It is a full structured data layer that supports:

  • Type-safe indexing on keyword, integer, float, bool, geo, datetime, UUID, and text fields
  • Boolean filtering with nested must (AND), should (OR), and must_not (NOT) clauses
  • Geographic queries with bounding box and radius filtering
  • Full-text search with configurable tokenization, stemming, stopword removal, and ASCII folding
  • Faceted aggregation for counting distinct values
  • Nested object filtering for traversing JSON hierarchies

The payload types matter because they determine what filtering operations are available. A range filter on a string field returns nothing. A geo filter on an integer field returns nothing. Qdrant does not guess — it enforces type discipline.

# This filter narrows vector search to EU regulatory documents
# published after January 2026, tagged with "AI"
from qdrant_client import QdrantClient, models

client = QdrantClient(url="http://localhost:6333")

client.query_points(
    collection_name="documents",
    query=[0.1, 0.45, 0.67, ...],
    query_filter=models.Filter(
        must=[
            models.FieldCondition(
                key="region",
                match=models.MatchValue(value="EU"),
            ),
            models.FieldCondition(
                key="effective_date",
                range=models.Range(gte="2026-01-01T00:00:00Z"),
            ),
            models.FieldCondition(
                key="tags",
                match=models.MatchAny(any=["AI"]),
            ),
        ]
    ),
    limit=10,
)

This is where vector databases diverge from vector libraries. FAISS gives you nearest neighbors. Qdrant gives you nearest neighbors within a filtered, typed, indexed data layer.

Hybrid Search: The Architecture Most Teams Are Missing

The single most common retrieval mistake is relying on dense vectors alone. Dense embeddings capture semantic similarity. They are terrible at exact keyword matching. If a user searches for “SOP-2024-0187” or “ICH Q9(R1)” or a specific product code, dense vectors will return semantically related documents that do not contain the exact term.

Hybrid search solves this by running two parallel retrieval paths and fusing the results.

                    ┌─────────────────────┐
                    │   QUERY: "ICH Q9"   │
                    └─────────┬───────────┘
                              │
              ┌───────────────┴───────────────┐
              │                               │
    ┌─────────▼──────────┐          ┌─────────▼──────────┐
    │  DENSE PATH        │          │  SPARSE PATH       │
    │  Semantic Embedding│          │  BM25 Embedding    │
    │  → HNSW Search     │          │  → Sparse Index    │
    │  → 100 candidates  │          │  → 100 candidates  │
    └─────────┬──────────┘          └─────────┬──────────┘
              │                               │
              └───────────────┬───────────────┘
                              │
                    ┌─────────▼──────────┐
                    │  FUSION            │
                    │  RRF or DBSF       │
                    │  → Final Top 10    │
                    └────────────────────┘

Qdrant implements this through the prefetch API. A hybrid query specifies multiple prefetch sub-queries, each targeting a different named vector, then fuses the results:

client.query_points(
    collection_name="documents",
    prefetch=[
        models.Prefetch(
            query=models.Document(
                text="ICH Q9 risk-based approach",
                model="sentence-transformers/all-minilm-l6-v2"
            ),
            using="dense-embedding",
        ),
        models.Prefetch(
            query=models.Document(
                text="ICH Q9 risk-based approach",
                model="Qdrant/bm25",
            ),
            using="bm25-sparse",
        ),
    ],
    query=models.FusionQuery(fusion=models.Fusion.RRF),
    limit=10,
)

Two fusion methods are available:

Method How It Works Best For
RRF (Reciprocal Rank Fusion) Boosts items appearing near the top in multiple result sets. Score = Σ(1/(k + rank)). Most cases; simple, robust, no tuning
DBSF (Distribution-Based Score Fusion) Normalizes scores statistically before combining When score scales from different sources are wildly different

The BM25 model runs server-side inside the Qdrant cluster. You do not need a separate embedding service for lexical search. Specify model="Qdrant/bm25" and Qdrant generates the sparse vectors internally.

Multi-Stage Retrieval: Cheap First, Expensive Second

For large collections, running a high-dimensional search on every query is expensive. Multi-stage retrieval uses the prefetch API to run a cheap first pass, then re-rank the survivors with a more expensive model.

The Matryoshka model pattern is the canonical example:

client.query_points(
    collection_name="documents",
    prefetch=models.Prefetch(
        query=models.Document(
            text="GxP validation requirements",
            model="openai/text-embedding-3-small",
            options={"mrl": 64},  # 64-dim reduced vector
        ),
        using="small-embedding",
        limit=1000,  # cheap pass: 1000 candidates
    ),
    query=models.Document(
        text="GxP validation requirements",
        model="openai/text-embedding-3-small",
        # full 1536-dim vector
    ),
    using="full-embedding",
    limit=10,  # expensive pass: final 10
)

First stage: 64-dimensional search across the entire collection. Fast, low memory, broad recall. Second stage: full 1536-dimensional re-ranking on the top 1000 candidates. Precise, expensive, narrow.

Qdrant Cloud supports Matryoshka natively — a single inference request generates the full vector, and Qdrant truncates it to the requested dimension. No extra API calls.

Score Boosting: Injecting Business Logic

Vector similarity is not the only signal that matters. A document from 2026 is probably more relevant than one from 2019. A title match is more valuable than a body match. An authoritative source outranks a blog post.

Formula Query (v1.14+) lets you re-score results with custom expressions:

client.query_points(
    collection_name="documents",
    prefetch=models.Prefetch(
        query=[0.1, 0.45, 0.67],
        limit=50
    ),
    query=models.FormulaQuery(
        formula=models.SumExpression(sum=[
            "$score",
            models.MultExpression(mult=[
                0.5,
                models.FieldCondition(
                    key="doc_type",
                    match=models.MatchAny(any=["title", "heading"])
                )
            ]),
            models.MultExpression(mult=[
                0.3,
                models.FieldCondition(
                    key="authority_score",
                    range=models.Range(gte=0.8)
                )
            ]),
        ])
    )
)

This takes the base vector similarity score, adds 0.5 if the document is a title or heading, and adds 0.3 if the authority score is above 0.8. Conditions evaluate to 1.0 (true) or 0.0 (false), so the formula is effectively: score + 0.5 * is_title + 0.3 * is_authoritative.

Available expressions: $score, payload keys, constants, sum, mult, div, abs, pow, sqrt, neg, log1p, sigmoid, and nested conditions.

Recommendation and Discovery APIs

Beyond standard similarity search, Qdrant provides two exploration APIs.

Recommendation API takes positive and negative examples and finds items similar to the positives but dissimilar to the negatives. Two strategies:

  • average_vector (default): Averages positives and negatives into a single query vector. Formula: avg_positive + avg_positive - avg_negative. Fast — same cost as regular search.
  • best_score: Scores each candidate against each example individually, takes the best match per candidate. More nuanced but slower.

Discovery Search uses context pairs (positive, negative) as a one-shot training set. It learns a direction in vector space from the context and searches along it. Useful for interactive exploration where the user provides “more like this, less like that” signals.

Filtering: The Query Planner Under the Hood

When you combine vector search with payload filters, Qdrant’s query planner makes a critical decision: search the vector index first and filter after (post-filter), or filter first and search only the matching subset (pre-filter).

The decision is based on filter cardinality estimation using payload index statistics. A filter matching 1% of points should use pre-filtering. A filter matching 99% should use post-filtering. The payload index provides the cardinality estimates that drive this decision.

This is why creating payload indexes before data ingestion matters. Without indexes, Qdrant cannot estimate cardinality and must fall back to less optimal strategies.

# Create payload index BEFORE upserting data
client.create_payload_index(
    collection_name="documents",
    field_name="region",
    field_schema=models.KeywordIndexParams(
        type=models.KeywordIndexType.KEYWORD,
        is_tenant=True,  # co-locates vectors by tenant
    ),
)

The is_tenant=true flag is not just a label. It tells Qdrant to physically co-locate vectors of the same tenant on disk, enabling sequential reads instead of random seeks. For multi-tenant applications, this can be a 5–10x improvement in query throughput.

Quantization: Trading Precision for Speed

Qdrant supports four quantization methods, each with different compression ratios and recall trade-offs:

Method Compression Recall Speed Best For
Scalar 4x High Moderate General purpose; well-established
TurboQuant 4-bit 8x High Fast Best recall-to-compression ratio
TurboQuant 2-bit 16x Moderate Very fast Large collections with tight memory
Binary 16–32x Moderate Fastest Embeddings with >1000 dimensions
Product (PQ) Up to 64x Lower Moderate Maximum compression priority

TurboQuant (v1.18+, developed by Google) applies a fast random rotation to vectors before compression. At 4-bit, it achieves double the compression of scalar quantization with comparable recall. At 2-bit and below, it consistently outperforms binary quantization in recall while matching its speed.

Quantization can be applied globally on the collection or per-vector. The always_ram parameter keeps quantized vectors in RAM even when original vectors are on disk. The rescore parameter re-ranks quantized results with original vectors for higher precision.

Qdrant Edge is the most underappreciated feature in the ecosystem. It is an embedded vector search engine that runs inside your application process — no server, no background services, no network calls.

Think of it as SQLite, but for vector similarity search.

┌──────────────────────────────────────────────────────┐
│                EDGE DEVICE                           │
│                                                      │
│  ┌────────────────────────────────────────────────┐  │
│  │           APPLICATION PROCESS                  │  │
│  │                                                │  │
│  │  ┌──────────────────────────────────────────┐  │  │
│  │  │         QDRANT EDGE SHARD                │  │  │
│  │  │                                          │  │  │
│  │  │  ┌──────────────┐  ┌──────────────────┐  │  │  │
│  │  │  │ Vector Store │  │ Payload Store    │  │  │  │
│  │  │  └──────────────┘  └──────────────────┘  │  │  │
│  │  │  ┌──────────────┐  ┌──────────────────┐  │  │  │
│  │  │  │ HNSW Index   │  │ Payload Index    │  │  │  │
│  │  │  └──────────────┘  └──────────────────┘  │  │  │
│  │  │  ┌──────────────┐  ┌──────────────────┐  │  │  │
│  │  │  │ BM25 Engine  │  │ WAL              │  │  │  │
│  │  │  └──────────────┘  └──────────────────┘  │  │  │
│  │  └──────────────────────────────────────────┘  │  │
│  │                                                │  │
│  │  ┌──────────────────────────────────────────┐  │  │
│  │  │  FastEmbed (local models, no network)    │  │  │
│  │  └──────────────────────────────────────────┘  │  │
│  └────────────────────────────────────────────────┘  │
│                                                      │
│  Optional: ──────── Sync ──────────► Qdrant Server   │
└──────────────────────────────────────────────────────┘

Available as Python bindings (qdrant-edge-py) and Rust crate (qdrant-edge), Qdrant Edge supports:

  • All Qdrant quantization methods (scalar, product, binary, TurboQuant)
  • Schema evolution — add or remove named vectors on a running shard
  • Full payload indexing and filtering
  • A built-in BM25 embedder for on-device keyword search (no internet required)
  • Snapshot-based synchronization with a central Qdrant Server

The dual-shard sync pattern is the recommended architecture for edge devices that need both local writes and server-synced data:

  1. Mutable Edge Shard — handles local data updates
  2. Immutable Edge Shard — mirrors server data via partial snapshots
  3. Merged queries — search both shards and combine results

This enables use cases like robots that build local perception indexes while syncing with a fleet-wide knowledge base, or kiosks that personalize locally while receiving catalog updates from the cloud.

On-Device Embeddings with FastEmbed

Pairing Qdrant Edge with FastEmbed enables fully offline vector search — embedding generation and retrieval with zero network dependency.

The provisioning workflow:

  1. Install fastembed and qdrant-edge-py on the device
  2. Download embedding models to a local cache directory (CLIP-based models for text and images)
  3. Create an Edge Shard with the correct vector dimensions
  4. Generate embeddings and upsert points locally
from fastembed import ImageEmbedding, TextEmbedding

# Download once, use forever (offline)
text_model = TextEmbedding(
    model_name="Qdrant/clip-ViT-B-32-text",
    cache_dir="./models",
    local_files_only=True
)

image_model = ImageEmbedding(
    model_name="Qdrant/clip-ViT-B-32-vision",
    cache_dir="./models",
    local_files_only=True
)

The Edge BM25 embedder is compatible with server-side BM25 — same token IDs, same scoring formula. You can initialize an Edge Shard from a server snapshot and query it with locally generated BM25 vectors without re-indexing.

The Inference Pipeline: No Separate Embedding Stack

Qdrant’s inference API eliminates the need for a separate embedding service in many deployments. Four options, from closest to farthest:

Option Where It Runs Latency Models
Qdrant Cluster BM25 Inside your Qdrant cluster Lowest BM25 sparse
Qdrant Cloud Inference Qdrant-managed infrastructure Low (in-cluster for queries) Growing list, some free
External Providers (via Qdrant Cloud) OpenAI, Cohere, Jina, OpenRouter API latency Full provider catalog
Client-side (FastEmbed) Your own machine Local GPU/CPU FastEmbed model catalog

The key insight: you can mix inference sources in a single request. One upsert can generate image embeddings via Jina AI, text embeddings via Qdrant Cloud, and BM25 embeddings via the local cluster — all in a single API call.

client.upsert(
    collection_name="documents",
    points=[
        models.PointStruct(
            id=1,
            vector={
                "image": models.Document(
                    image="<url>",
                    model="jina-clip-v2"
                ),
                "text": models.Document(
                    text="Document description",
                    model="all-minilm-l6-v2"
                ),
                "bm25": models.Document(
                    text="Document description",
                    model="Qdrant/bm25"
                ),
            },
        )
    ],
)

Storage Architecture: Segments and Memory Tiers

Qdrant divides collection data into segments — independent units with their own vector storage, payload storage, indexes, and ID mapper.

Two segment types:

  • Appendable — full read/write/delete (mutable data)
  • Non-appendable — read/delete only (optimized immutable segments after compaction)

Vector storage uses memory-mapped files with two tiers:

Tier Behavior When to Use
cached (default) Pre-loads mmap into disk cache on startup Enough RAM for vectors; fast first query
cold No pre-loading; OS caches pages on access Large collections on fast disks; accept slow first query

For collections exceeding RAM, set on_disk=true on the HNSW index to store the graph structure on disk as well.

Operational Patterns

Bulk Upload Optimization

Technique Impact
Batch 64–256 points per request Minimizes per-request overhead
2–4 parallel upload threads Saturates write capacity
2–4 shards per collection Parallelizes across write workers
Create payload indexes first Builds HNSW links during ingestion, not after
Set vectors to on_disk/cold tier Prevents OOM on large datasets

Multitenancy: One Collection, Many Tenants

# Step 1: Create tenant index
client.create_payload_index(
    collection_name="shared",
    field_name="tenant_id",
    field_schema=models.KeywordIndexParams(
        type=models.KeywordIndexType.KEYWORD,
        is_tenant=True,
    ),
)

# Step 2: Every query includes tenant filter
client.query_points(
    collection_name="shared",
    query=[0.1, 0.2, 0.3],
    query_filter=models.Filter(
        must=[models.FieldCondition(
            key="tenant_id",
            match=models.MatchValue(value="tenant_42"),
        )]
    ),
)

One collection with is_tenant=true outperforms per-tenant collections in almost every scenario. The physical co-location of tenant data enables sequential disk reads and better cache utilization.

Low-Latency Checklist

  1. Payload indexes on all filtered fields
  2. Horizontal replicas for read distribution
  3. Delayed fan-out (v1.17+) for slow-replica mitigation
  4. Quantization with rescoring for memory-constrained deployments
  5. Multi-stage retrieval with Matryoshka for large collections
  6. score_threshold for early termination
  7. Block queries on unindexed fields (reject at API boundary)

The Bottom Line

Qdrant is not a simple vector store with an HTTP API. It is a retrieval engine with:

  • Hybrid search combining dense, sparse, and late-interaction vectors with RRF/DBSF fusion
  • A structured data layer with typed indexes, Boolean filtering, geo queries, and full-text search
  • Custom scoring via Formula Query to inject business logic into similarity ranking
  • Edge deployment as an embedded engine with on-device BM25 and FastEmbed integration
  • Built-in inference spanning BM25, Cloud-hosted models, and external providers
  • Quantization from 4x scalar to 64x product, with Google’s TurboQuant bridging the gap

If you are running Qdrant as a dumb nearest-neighbor index and calling client.search() with a raw vector, you are leaving most of its value on the table. Start with hybrid search. Add payload filtering. Layer in score boosting. Deploy an edge shard. The retrieval quality improvement will dwarf anything you can achieve by swapping embedding models.

Research notes: [[Qdrant Vector Database - Comprehensive User Manual Report]]