What is RAG and How Does it Work?
RAG stands for Retrieval-Augmented Generation. It's a technique that transforms any document or database into an AI system capable of answering questions accurately.
The Problem RAG Solves
AI models like ChatGPT and Claude have two problems:
1. **Training data cutoff** — they don't know events after their last training
2. **Hallucination** — they may invent incorrect information
RAG solves this by connecting the model to your own data.
How RAG Works
Phase 1: Indexing
1. Split your documents into small pieces (Chunks)
2. Convert each piece to a mathematical vector (Embedding)
3. Store vectors in a vector database
Phase 2: Retrieval
1. User types their question
2. Question is converted to a vector
3. Search for the most similar pieces in the database
4. Retrieve the most relevant pieces
Phase 3: Generation
1. Send question + retrieved pieces to LLM
2. LLM generates an answer based on the real context
Practical Example
Without RAG:
Question: What is our store's return policy?
AI Answer: I'm sorry, I don't know your specific store's policy.
With RAG:
Question: What is our store's return policy?
(RAG finds page 12 from the policies document)
AI Answer: According to your policy, products can be returned within 30 days with receipt...
How to Build a Simple RAG System
from langchain.document_loaders import PyPDFLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.vectorstores import Chroma
from langchain.chains import RetrievalQA
# 1. Load document
loader = PyPDFLoader("document.pdf")
docs = loader.load()
# 2. Split text
splitter = RecursiveCharacterTextSplitter(chunk_size=500)
chunks = splitter.split_documents(docs)
# 3. Create vector database
vectorstore = Chroma.from_documents(chunks, embedding)
# 4. Build RAG chain
qa = RetrievalQA.from_chain_type(
llm=llm,
retriever=vectorstore.as_retriever()
)
# 5. Query
answer = qa.run("What are the main terms?")