Curriculum

Generative AI · Retrieval-Augmented Generation

RAG Architecture

RAG grounds LLM outputs in retrieved external knowledge, reducing hallucination. Advanced variants add semantic caching, hybrid BM25+dense retrieval, and agentic retrieval loops.

The idea

Retrieval-Augmented Generation (RAG) decouples knowledge storage from model parameters. A retriever fetches relevant chunks from a knowledge base; the LLM uses these as context in its prompt to generate a grounded answer.

The indexing pipeline: chunk documents → embed chunks → store in a vector database (Pinecone, Weaviate, pgvector). At query time: embed the query → retrieve top-k similar chunks by cosine similarity → prepend chunks to LLM prompt.

Hybrid search combines dense (embedding) and sparse (BM25 keyword) retrieval and fuses scores. It outperforms either alone, especially on rare keywords. Semantic caching stores previous query-answer pairs; similar future queries hit the cache instead of recomputing.

score(q,d) = λ·dense_sim(q,d) + (1−λ)·BM25(q,d)

Hybrid retrieval score: weighted combination of dense embedding similarity and BM25 keyword score.

What to hold onto

  • Chunking StrategiesFixed-size vs. sentence-boundary vs. semantic chunking. Chunk too small: insufficient context. Too large: exceeds LLM context window and dilutes signal.
  • RerankingUse a cross-encoder reranker (e.g., Cohere Rerank) on the top-50 retrieved chunks to re-order by relevance before feeding to the LLM. Improves faithfulness.
  • Agentic RAGAn LLM agent decides when to retrieve, what to query, and whether to retrieve again. More powerful but latency-sensitive.

Live: RAG Pipeline & Vector Context Retrieval

Input query $\to$ Dense Embedding $\to$ Top-$K$ Semantic Search $\to$ Augmented Prompt Context injection.

Augmented Context Assembled

Quick answers

What is Retrieval-Augmented Generation (RAG)?

RAG is a technique that enhances LLM outputs by first retrieving relevant documents from an external knowledge base and including them in the model's context. This grounds responses in factual sources, reduces hallucination, and enables LLMs to answer questions about information not seen during pre-training.

What is hybrid retrieval in RAG?

Hybrid retrieval combines dense vector search (embedding similarity) and sparse keyword search (BM25) and fuses their scores. Dense search excels at semantic understanding; sparse search handles exact keyword matches. Combining them outperforms either method alone, particularly for domain-specific queries.