AI glossary Data & Retrieval
What is retrieval in RAG and search systems?
← All glossary termsThe search step that fetches the documents or passages most relevant to a query, before they are shown or passed to a model.
Explained
Retrieval is the step in RAG and in search systems that finds, among a large collection, the documents or text passages most relevant to a query. It usually returns a small ranked list, the top-k results, from an index built in advance. It can work by matching keywords (sparse retrieval), by comparing embeddings in a vector space (dense retrieval) or by combining both. In a RAG system, what retrieval returns is all the model sees of your knowledge, so its quality sets the ceiling for the quality of the answer.
Sparse, dense and hybrid
- Sparse retrieval matches the words in the query with the words in the documents, weighted by how distinctive they are. BM25 is the classic method. It is fast, transparent and good with exact terms such as codes and names.
- Dense retrieval turns queries and passages into embeddings and finds the nearest ones in a vector index. It finds passages that mean the same thing even when they use different words.
- Hybrid retrieval runs both and merges the results. In enterprise content, where exact terms and paraphrases both matter, it is often the safer default.
Reranking and filtering
First-stage retrieval is built for speed over large collections, so it returns a generous set of candidates. A reranker, usually a model that reads the query and each candidate together, then orders them more precisely and keeps the best few.
Filters are applied around this: by date, document type, language and, above all, by the user's permissions, so that retrieval never returns content the user is not allowed to see.
How to tell if retrieval works
- Build a test set of real questions with the passages that answer them.
- Measure recall at k: how often the right passage is among the first k results.
- Measure ranking quality, for example with mean reciprocal rank.
- Evaluate retrieval separately from generation, so you know which one to fix.
- Review failures by hand: they often point to bad chunking, missing documents or vocabulary gaps.
Frequently asked questions
What does retrieval mean in RAG?
It is the R in retrieval-augmented generation: the step that searches your documents for passages relevant to the question. Those passages are then given to the language model as context for its answer.
What is BM25?
BM25 is a ranking function for keyword search. It scores documents by how often the query terms appear in them, adjusted for how rare each term is across the collection and for document length. It remains a strong baseline and a common part of hybrid search.
What is top-k in retrieval?
Top-k is the number of results the retriever returns, for example the five or ten most relevant passages. A higher k raises the chance of including the right passage but adds noise and cost when the passages are sent to a model.
Is dense retrieval always better than keyword search?
No. Dense retrieval is better at matching meaning, but keyword search is often better for exact identifiers, rare terms and names. That is why many systems combine both and rerank the merged results.
What is a reranker?
A reranker is a model that takes the query and a set of candidate passages and orders them by relevance more precisely than the first retrieval stage. It is slower per passage, so it is applied only to the shortlist.
Decision guides that use this term
How to put it to work
Where it shows up at Thinkia
Further reading
Related terms
Sources
- Manning, Raghavan & Schütze · Introduction to Information Retrieval (Stanford NLP, online edition)
- Karpukhin et al. · Dense Passage Retrieval for Open-Domain Question Answering (arXiv)
- Lewis et al. · Retrieval-augmented generation for knowledge-intensive NLP tasks (arXiv)
Updated: 2026-10-21