RAG (Retrieval-Augmented Generation)
What is RAG (Retrieval-Augmented Generation)?
RAG (Retrieval-Augmented Generation) is the technique of searching a business's own documents, knowledge base, or database for relevant passages at the moment a question is asked, then feeding those passages into a language model's prompt so its answer is grounded in that specific content instead of relying only on what the model memorized during training. RAG is the standard way to make an AI agent answer accurately about a company's own products, policies, or account data.
More detail
A RAG pipeline has two stages. At index time, documents are split into passages, embedded into a vector index, and periodically re-indexed as content changes. At query time, the pipeline retrieves the passages most semantically similar to the incoming question and hands them to the language model, which composes the final answer from those sources.
Where a chain-of-contact AI agent runs on Orbit, the knowledge base that material feeds the voice and messaging agents the same way: an agent answering a call resolves the caller's question against current product, policy, and account data rather than a frozen training snapshot — without which it could hold only the small, fixed set of scripted replies.
RAG reduces hallucination risk by grounding answers in retrieved source text, but the quality of retrieval — how well the right passages are found — matters as much as the model generating the final answer. A good pipeline also returns 'I don't know' when retrieval finds nothing, instead of letting the model fill the gap from its training.
RAG and fine-tuning are complementary, not rivals: fine-tuning shapes how a model speaks and behaves, while RAG decides what facts that behavior is anchored to. Most production agents pair both — a tuned model answering over retrieved passages.
Because the retrieval scope is a tenant's own knowledge base, a RAG agent only ever pulls passages from the tenant's indexed documents — it doesn't roam into another tenant's data or the open internet for an answer.
Frequently asked
- How does retrieval-augmented generation work, step by step?
- Documents are split into passages and embedded into a searchable index ahead of time. When a question arrives, the pipeline retrieves the passages most similar to that question, adds them to the language model's prompt as source material, and the model composes its answer from those passages rather than relying only on what it memorized during training.
- Why use RAG instead of just fine-tuning a model on company data?
- RAG lets an AI agent answer from current, easily updated source documents without retraining the underlying model every time content changes, which is faster and cheaper than fine-tuning for information that updates frequently, like pricing or policy documents. Fine-tuning still helps shape tone and behavior; most production agents combine both.
- Does RAG eliminate hallucination entirely?
- No — it significantly reduces it by grounding answers in retrieved source text, but if retrieval surfaces the wrong or incomplete passages, the model can still generate an inaccurate answer, so retrieval quality is a critical factor. A well-built pipeline also abstains when it finds nothing relevant instead of guessing.
- What content belongs in a RAG knowledge base for a support agent?
- The texts that already answer customer questions: product documentation, policy pages, past support resolutions, and internal playbooks. Anything a customer can legitimately ask about — an agent grounded in those materials stops deflecting to 'contact support' on answerable questions.
See also
Build it on Orbit
Voice, messaging, email, video, and AI agents on one platform and one pay-as-you-go bill. Start free — no credit card required.