Skip to main content
RAG
AI LabRAG Strategy Lab

RAG Strategy Lab

Compare retrieval strategies on diverse datasets: from portfolio questions to recipe searches to movie analysis

Claude Haiku 4.5Vector SearchHybrid (BM25)Re-ranking

What is this?

An interactive comparison of RAG (Retrieval-Augmented Generation) strategies across three diverse knowledge bases (73 portfolio docs, 381 recipes, 4K movie reviews). Compare quality, speed, and cost trade-offs.

Educational tool: Compare Basic (vector only), Hybrid (vector + keywords), Re-ranking (LLM judgment), and No RAG baseline.

Best viewed on desktop for full side-by-side comparisons and metrics tables.

β†’ Learn about strategies below, then try live comparisons.

This reveals how RAG systems work and when each strategy excels!

My RAG Journey: Started with the AI Assistant on my homepage (Basic RAG), then built this lab to compare strategies and understand trade-offs.

Select Knowledge Base

(Loading counts...)

Select Strategies to Compare

(2 selected, 2-3 allowed)

What This Knowledge Base Shows73 Docs

No single strategy wins every question - This is a small, focused set (~73 documents about one person's career), so most questions work with any search strategy. The differences show up on specific kinds of questions.

Exact codes like "HL7" get blurred by meaning search, so Hybrid catches them. Questions where many documents look alike ("key technical skills") need Re-ranking to pick the right one. And a question with no single document holding the answer ("certifications") failed for every strategy until one summary document was added. Try the scenarios below and check which sources each strategy found.

When Does Each Strategy Win?

Industry best practices - click to see side-by-side comparison

Basic RAG

🎯 Search by meaning

Best for:

  • βœ“Semantic/conceptual queries: "Tell me about healthcare experience"
  • βœ“Natural language questions: "How did she scale CM-SHARE?"
  • βœ“Small, focused knowledge bases: Single domain, clear boundaries
Speed:
5/5
Cost:
1/5
Quality:
3/5

Why it wins: With ~73 documents about one person's career, most questions land on the right document by meaning alone. It struggles with short codes (HL7) and with picking one document out of many similar ones (try the scenarios below).

Examples: Focused chatbots, small document collections, first prototypes

When scope is contained, semantic search alone is often sufficient. See the Embedding Visualization to see how 73 documents form clean, separated clusters.

Hybrid RAG

🏭 Meaning + keywords

Best for:

  • βœ“Exact keyword matches: Company names, acronyms (FHIR, CM-SHARE)
  • βœ“Technical terms: "HL7", "FDA compliance"
  • βœ“Large, diverse knowledge bases: Thousands+ documents across many topics
Speed:
4/5
Cost:
2/5
Quality:
4/5

Why it wins: At scale (Google, enterprise wikis), semantic similarity alone creates ambiguity. "Healthcare AI" could match hundreds of documents - keyword boost helps pinpoint the exact concept you need.

Examples: Search engines that offer both keyword and vector search (Elasticsearch, OpenSearch, Azure AI Search)

Hybrid is a common default for production RAG because real questions mix concepts with exact names and codes. In this lab, the HL7 scenario shows it: meaning search misses the exact term, hybrid catches it.

Re-ranking

⚑ AI judge

Best for:

  • βœ“Subjective queries: "Best for...", "Recommend..."
  • βœ“Complex context: Multi-part questions, synthesis needed
  • βœ“High-stakes decisions: Worth the extra cost/latency
Speed:
2/5
Cost:
4/5
Quality:
5/5

Why it wins: LLM understands query intent and provides nuanced relevance judgment. Cast a wide net with retrieval, then intelligently re-rank with AI.

Examples: Reranking models and services (Cohere Rerank, Voyage rerank) or an LLM scoring step like this lab uses

Use it when many documents look alike and only one holds the answer. In this lab, the skills scenario shows it: only re-ranking finds the skills summary. It adds an extra AI call, so it is slower and costs more.

No RAG

πŸ“Š Baseline Only

Best for:

  • βœ“General knowledge: Questions within LLM training data
  • βœ“Baseline testing: Measuring RAG value
  • βœ“Demonstrating hallucination: Shows why RAG is needed
Speed:
5/5
Cost:
3/5
Quality:
1/5

Why it wins: It doesn't. This is for comparison only.

Examples: Proof-of-concept demos, A/B testing, educational examples

Not production-ready for domain-specific knowledge. LLMs hallucinate confidently when lacking context. Always use RAG for factual, up-to-date, or specialized information.

Quick decision: Small dataset? β†’ Basic RAG. Large dataset or technical terms? β†’ Hybrid RAG. High-stakes queries? β†’ Re-ranking. Proving RAG value? β†’ Compare with No RAG.

πŸ”¬ Research Frontier: Advanced Strategies

Techniques that shine at scale - worth exploring for large systems

These advanced techniques add complexity but can improve retrieval quality for large-scale systems (10,000+ documents). For smaller knowledge bases like this portfolio, the strategies above are sufficient.

πŸ”„Query Expansion
Paper

Multi-Query Retrieval - Generate query variations using LLM, retrieve for each, merge results. Improves recall by capturing different phrasings of the same intent.

πŸ§ͺHyDE
Paper

Hypothetical Document Embeddings - Generate a hypothetical answer, embed that instead of the question. Aligns query embeddings with document space. Helps most on abstract queries.

πŸ“ŠRRF
Paper

Reciprocal Rank Fusion - Combine rankings from multiple retrieval methods using a proven formula. Simple and hard to break, because it uses ranks instead of raw scores.

⚑ColBERT
GitHub

Late Interaction - Token-level matching between query and documents. Balances quality of cross-encoder with speed of bi-encoder. Industry standard for large-scale re-ranking.

πŸ¦–RAPTOR
Paper

Recursive Summarization - Build hierarchical summaries of documents for multi-level retrieval. Best for very long documents or when you need both detail and overview.

πŸ’‘ When to explore these: If you have 10,000+ documents, see diminishing returns from basic strategies, or need sub-100ms latency at scale.

β–Ά

🎯 Try It Yourself:

Pick a scenario below. Each one has an answer key, so every strategy shows a βœ“ or βœ— for whether it found the right source.

Then read the answers. A confident answer built on the wrong sources is the most common way RAG fails.

Pick a scenario, or type your own question below

Scenarios have an answer key, so results show whether each strategy found the right source. Your own questions don't, so check the sources yourself.

Behind the build