Skip to main content

AI glossary Models & Architecture

What is retrieval-augmented generation (RAG)?

RAG (Retrieval-Augmented Generation)

← All glossary terms
In short

A technique that retrieves relevant passages from a knowledge base and gives them to a language model as context, so its answer rests on those sources.

Explained

Retrieval-augmented generation (RAG) combines a search step with a language model. When a question arrives, the system looks up the most relevant passages in a document collection, usually through a vector index, and passes them to the model together with the question. The model then writes its answer from that material instead of relying only on what it learnt in training. This keeps answers tied to current, private sources, lets the system cite where each answer came from and reduces hallucinations, without retraining the model.

How RAG works

  • Ingestion: documents are split into chunks and each chunk is turned into an embedding, a vector that captures its meaning.
  • Indexing: the vectors are stored in a vector database or search engine, often alongside a keyword index.
  • Retrieval: the question is embedded the same way and the closest chunks are fetched, sometimes reordered by a reranker.
  • Generation: the model receives the question plus the retrieved chunks and writes the answer, ideally citing them.

RAG or fine-tuning?

RAG changes what the model can see at the moment it answers; fine-tuning changes the model itself. RAG suits knowledge that changes often, must stay under access control or needs to be cited. Fine-tuning suits a stable style, format or narrow task that the model should perform without long instructions.

The two are not exclusive. Many production systems use a general model, retrieval for the facts and, only where it pays off, a fine-tuned model for tone or structure.

Limits and what to watch

  • Retrieval quality sets the ceiling: if the right passage is not retrieved, the model cannot use it.
  • Chunking, metadata and permissions need care, or users see answers built from documents they should not access.
  • The model can still misread or over-generalise the context, so answers need evaluation against real questions.
  • Corpus-wide questions ("what are the main themes?") are hard for plain RAG and may need summarisation or graph-based approaches.

Frequently asked questions

What does RAG stand for in AI?

RAG stands for retrieval-augmented generation. The term comes from a 2020 paper by Patrick Lewis and colleagues at Facebook AI Research, University College London and New York University.

Does RAG eliminate hallucinations?

No. It reduces them by giving the model relevant sources, but the model can still ignore, misread or go beyond the retrieved text. Good prompts, citations and systematic evaluation are still needed.

Do I need a vector database for RAG?

Not always. Vector search is the most common retrieval method, but RAG can use keyword search, a hybrid of both, SQL queries or a knowledge graph. The right choice depends on the data and the questions people ask.

What is the difference between RAG and fine-tuning?

RAG adds information at query time by retrieving documents; fine-tuning adjusts the model's weights with extra training. RAG is easier to update and audit, while fine-tuning is better for teaching a consistent behaviour or format.

What is agentic RAG?

Agentic RAG lets an AI agent decide when and how to search: it can reformulate the query, search several sources, check what it found and search again. It handles complex questions better than a single retrieval step, at the cost of more latency and more calls to the model.

Decision guides that use this term

How to put it to work

Where it shows up at Thinkia

Further reading

Sources

Updated: 2026-10-21

Want to see how this applies to your organisation?

Talk to an AI Expert