qdrant-search-quality-diagnosis - SKILL.md Agent Skill

name: qdrant-search-quality-diagnosis description: "Diagnoses Qdrant search quality issues. Use when someone reports 'results are bad', 'wrong results', 'not relevant results', 'missing matches', 'recall is low', 'approximate search worse than exact', 'which embedding model', 'quality dropped after quantization', 'how to measure retrieval quality', 'build a golden set', 'ground truth dataset', or 'how to score recall@k'. Also use when search quality degrades without obvious changes."

How to Diagnose Bad Search Quality

Before tuning, establish baselines. Use exact KNN as ground truth, compare against approximate HNSW. Target >95% recall@K for production.

Don't Know What's Wrong Yet

Use when: results are irrelevant or missing expected matches and you need to isolate the cause.

For a no-code quick check, use the Web UI's ANN Recall tab to compare approximate vs exact recall@k Web UI ANN Recall
For the same comparison in code (CI gating, regression tests), run each query twice — once approximate, once with exact=true — and compute recall@k from the overlap ANN recall in CI
Exact search bad = model or search pipeline problem. Exact good, approximate bad = tune HNSW.
Check if quantization degrades quality (compare with and without)
Check if filters are too restrictive (then you might need to use ACORN)
If duplicate results from chunked documents, use Grouping API to deduplicate Grouping

Payload filtering and sparse vector search are different things. Metadata (dates, categories, tags) goes in payload for filtering. Text content goes in sparse vectors for search.

Approximate Search Worse Than Exact

Use when: exact search returns good results but HNSW approximation misses them.

Increase hnsw_ef at query time Search params
Increase ef_construct (200+ for high quality) HNSW config
Increase m (16 default, 32 for high recall) HNSW config
Enable oversampling + rescore with quantization Search with quantization
ACORN for filtered queries (v1.16+) ACORN

Binary quantization requires rescore. Without it, quality loss is severe. Use oversampling (3-5x minimum for binary) to recover recall. Always test quantization impact on your data before production. Quantization

Wrong Embedding Model

Use when: exact search also returns bad results.

Check Qdrant team recommendations on how to choose an embedding model.

Test top 3 MTEB models on 100-1000 sample queries Hosted Qdrant inference. Score them against a labeled set to compare apples to apples Measuring Retrieval Relevance.

Unoptimized Search Pipeline

Use when: exact search also returns bad results and model choice is confirmed by user.

Optimize search according to advanced search-strategies skill.

Need a Labeled Baseline to Score Recall, MRR, or NDCG

Use when: user has no golden set, asks "how do I know if my search is good?", or needs to gate releases on a retrieval metric.

Build a labeled query set — human, log-based, or LLM-synthetic — and score retrieval with ranx Measuring Retrieval Relevance
Pick the metric by usage: Recall@k for RAG, MRR/Hits@1 for single-answer, NDCG@k for re-ranking Choosing the metric
For full RAG pipelines, also score generation with Ragas and use the retrieval-vs-generation 2x2 to isolate regressions Pipeline Output Quality
Gate CI on a per-metric threshold to catch regressions from embedding-model swaps, prompt changes, or index config changes

What NOT to Do

Tune Qdrant before verifying the model is right for the task (most quality issues are model issues)
Use binary quantization without rescore (severe quality loss)
Set hnsw_ef lower than results requested (guaranteed bad recall)
Skip payload indexes on filtered fields then blame quality (HNSW can't traverse filtered-out nodes, and filterable HNSW is built only if payload indexes were set up prior)
Deploy without baseline recall or other search relevance metrics (no way to measure regressions)
Confuse payload filtering with sparse vector search (different things, different config)