What is RAG?

RAG stands for Retrieval-Augmented Generation. It’s a technique for making AI more useful by giving it access to information it wasn’t trained on.

The problem RAG solves is that AI models are trained on a fixed snapshot of data. They don’t know about anything that happened after their training cutoff, and they don’t know anything about your specific business, documents, or data. Ask a plain AI chatbot about your company’s internal policies and it will either make something up or tell you it doesn’t know.

RAG fixes this by adding a retrieval step. Before the AI generates a response, it searches through a set of documents (your docs, your database, your knowledge base) and pulls in the most relevant pieces. Those pieces get included in the prompt, so the AI can use them when forming its answer. The AI is still doing the generating, but it’s now working with real, relevant context instead of relying purely on what it learned during training.

The result is an AI that can answer questions about your specific data, stay up to date with new information, and cite sources rather than hallucinating. RAG is the foundation of most enterprise AI applications built today. Internal chatbots, document search, customer support tools. It’s all RAG under the hood.

RAG usually involves embeddings and a vector database to find semantically similar content quickly. But those are implementation details. The core idea is simple. Look something up, then answer.

I used RAG at my current company in a proof of concept to help with automatic configuration of our internal data models. ChatGPT has no idea what our application’s data model looks like, so we used RAG to feed it relevant examples at query time. The AI could then use those examples to generate configurations that actually matched our structure. It worked well and was a good illustration of exactly what RAG is for: giving the AI the context it needs to be useful in your specific domain.