RAG (Retrieval-Augmented Generation)

Definition

A technique that retrieves relevant documents from an external knowledge base and feeds them to a model alongside the user's query. RAG reduces hallucination and keeps responses grounded in current, factual data.

Why it matters

RAG is the most widely deployed architecture pattern in enterprise AI. It solves two fundamental problems: models do not know about your proprietary data, and models hallucinate when they lack information. By retrieving relevant documents and including them in the prompt, RAG gives the model factual grounding without requiring expensive fine-tuning. For enterprises, RAG means you can build accurate AI assistants over your internal knowledge bases, documentation, support tickets, and policy documents, in days rather than months. The catch: RAG quality depends entirely on retrieval quality. If you retrieve the wrong documents, the model will generate confident answers from irrelevant information.

In practice

The RAG pattern, introduced by Meta in 2020, has become the default architecture for enterprise AI assistants. The standard stack: embed documents into vectors using an embedding model, store in a vector database (Pinecone, Weaviate, Chroma), retrieve top-k relevant documents for each query, and include them in the prompt. Companies like Glean and Guru built products around RAG over enterprise knowledge bases. Advanced RAG techniques include: hybrid search (combining vector and keyword search), reranking retrieved results, iterative retrieval (the model asks follow-up questions), and agentic RAG (the model decides when and what to retrieve). RAG over internal data is now table stakes for enterprise AI vendors.