Retrieval

Definition

The process of finding and fetching relevant information from a knowledge base, database, or document store to provide context for an AI model. Retrieval quality is the single biggest determinant of RAG system performance.

Why it matters

Retrieval is where most RAG systems fail. You can have the best model in the world, but if you retrieve irrelevant documents, the output will be useless or misleading. Retrieval quality depends on: embedding model choice, chunking strategy, indexing approach, query formulation, and reranking. Each of these is a design decision that requires experimentation and tuning. The emerging consensus is that retrieval is an engineering discipline, not a commodity: the difference between a naive and a well-tuned retrieval pipeline is often a 2-3x improvement in end-to-end answer quality. Companies investing in retrieval infrastructure are building a durable competitive advantage.

In practice

Modern retrieval stacks combine multiple techniques: dense retrieval (vector search using embeddings), sparse retrieval (BM25/keyword matching), and learned reranking. Cohere's Rerank API and cross-encoder rerankers from Hugging Face improve retrieval precision by 20-40%. Pinecone, Weaviate, and Qdrant compete on retrieval performance and scale. Advanced techniques include: hypothetical document embeddings (HyDE), where the model generates a hypothetical answer and searches for similar documents; multi-query retrieval, where the model generates multiple search queries from a single user question; and agentic retrieval, where the model iteratively refines its search based on initial results.