Ask ChatGPT about your return policy and it will confidently make something up. That’s the problem retrieval-augmented generation (RAG) solves: it lets AI answer using your documents and data, with sources.
RAG in one sentence
Before the AI answers, it searches your knowledge base, pulls the most relevant passages, and uses them to write an answer — like an open-book exam instead of a memory test.
How it works
- Ingest — your documents (PDFs, Google Drive, Notion, help-desk articles, website) are split into chunks.
- Embed — each chunk is converted into a vector, a numeric representation of its meaning.
- Store — vectors go into a vector database such as Pinecone or Supabase pgvector.
- Retrieve — when a question comes in, the system finds the most relevant chunks by meaning, not just keywords.
- Generate — the LLM writes an answer using those chunks and cites them.
Why not just fine-tune a model?
Fine-tuning teaches a model style and patterns, but it’s a poor way to store facts that change. RAG is cheaper, updates instantly when documents change, supports permissions, and shows sources. For most business knowledge, RAG is the right tool.
Business use-cases
- Customer support bots that answer from your real docs.
- Internal assistants for SOPs, HR policies and onboarding.
- Sales enablement — instant answers from product specs and case studies.
- Legal & compliance — search contracts and regulations in plain English.
What makes RAG good (or bad)
Most failed RAG projects fail on retrieval, not the AI model. The difference is in chunking strategy, hybrid (vector + keyword) search, re-ranking, metadata filters and — critically — an evaluation set of real questions to measure accuracy before launch.
How RAG works, step by step
- Prepare: your documents are split into small passages.
- Index: each passage is turned into an embedding and stored in a vector database.
- Retrieve: when someone asks a question, the most relevant passages are found.
- Generate: the AI model writes an answer using those passages, ideally with sources.
RAG vs fine-tuning
| RAG | Fine-tuning | |
|---|---|---|
| Best for | Facts, documents, policies that change | Style, format, specialised behaviour |
| Updating knowledge | Add or edit documents | Retrain the model |
| Shows sources | Yes | No |
| Typical cost | Lower, ongoing small costs | Higher upfront |
Privacy implications of RAG systems
- Permissions: retrieval must respect who is allowed to see each document.
- Sensitive data: exclude or mask personal and confidential data you don’t need indexed.
- Vendors: use providers that don’t train on your data, with clear retention rules.
- Logs: questions and answers can contain sensitive information, so secure them too.
Common RAG use cases
- Internal knowledge assistants for policies, SOPs and manuals.
- Customer support answers grounded in help-center articles.
- Sales and proposal assistants that pull from past documents.
- Contract and document search for legal and finance teams.
What does it cost?
A production RAG system typically starts in the low thousands of dollars to build, plus modest monthly hosting and usage. See our RAG systems service or get a scoped quote.
See it in practice: RAG support agent for a B2B SaaS and Contract & precedent search for a corporate law firm.
Local pages: we deliver rag & knowledge ai for businesses in San Francisco, Boston and across the US.