AI decisions Architecture and technology
GraphRAG vs vector RAG: when does your knowledge assistant need a graph?
Start with vector RAG, ideally with hybrid search and reranking: it answers most questions that live in one or a few passages, and it is cheaper to build and keep fresh. Add a knowledge graph when the questions you must answer depend on relationships across many documents, such as who is connected to what, multi-step chains or themes across a whole corpus, and evaluation shows vector retrieval missing them. GraphRAG buys that reach with a costlier index, more maintenance and harder permission handling.
The options
Vector RAG
Documents are split into chunks, turned into embeddings and retrieved by semantic similarity to the question, often combined with keyword search and reranking.
GraphRAG
Entities and relationships are extracted from the sources into a knowledge graph, and retrieval follows or summarises that graph, usually alongside text chunks.
Side by side
| Criterion | Vector RAG | GraphRAG |
|---|---|---|
| Questions it answers well | Local questions whose answer sits in one or a few passages: a policy clause, a procedure, a definition. | Questions that connect entities across documents, need several hops, or ask for themes and patterns across the whole corpus. |
| Building the index | Chunking and embedding; relatively cheap and fast, even for large corpora. | Extracting entities and relationships, often with an LLM over every chunk; noticeably more compute, time and design work. |
| Keeping it fresh | New or changed documents are re-embedded and replace their old chunks. | Changes ripple through entities, relationships and any precomputed summaries; incremental updates need careful design. |
| Data modelling effort | Little upfront modelling; quality depends on chunking, metadata and retrieval tuning. | Needs decisions on entity types, relationships and entity resolution, either a defined ontology or a governed extraction prompt. |
| Explainability | Shows the passages used; it does not show why two facts are connected. | Can show the path through entities and relationships behind an answer, besides the source passages. |
| Access control | Permissions can be filtered per chunk, inherited from the source document. | Entities and summaries mix facts from many documents, so enforcing per-user permissions is harder and must be designed in. |
| Latency and cost per query | Usually low and predictable. | Varies with the query mode: graph traversal can be fast, while global summarisation over many communities can be slow and costly. |
| Typical failure mode | Misses answers that require connecting facts spread across documents, or retrieves similar but irrelevant passages. | Extraction errors and duplicate entities enter the graph and propagate into confident answers built on wrong links. |
Choose Vector RAG when…
- Most questions are about what a document says: policies, procedures, manuals, contracts one at a time.
- The corpus changes often and answers must reflect the latest version quickly.
- Users have different permissions and access must be enforced per document.
- You need a first useful version soon and want to learn from real questions before modelling anything.
Choose GraphRAG when…
- The value is in relationships: suppliers and their owners, products and components, cases and the people involved.
- Users ask multi-step questions whose answer is not written in any single passage.
- You need overviews across a large corpus, such as recurring themes or risks, not only lookups.
- The entities and relationships are fairly stable and you already have, or want, a governed data model for them.
- Evaluation on real questions shows vector retrieval failing precisely where connections matter.
When to combine them
In practice the choice is rarely exclusive. A common pattern keeps vector or hybrid retrieval as the default path and adds the graph for the questions that need it: the graph identifies the relevant entities and their neighbours, and text retrieval brings back the passages that prove each link. A router, or an agent with both as tools, decides per question. Introduce the graph for one domain where relationships carry the value, measure it against the same evaluation set, and widen it only if it earns its cost.
Common mistakes
- Building a knowledge graph of everything before knowing which questions it has to answer.
- Choosing GraphRAG because vector RAG underperformed, when the real gap was chunking, metadata, hybrid search or reranking.
- Ignoring entity resolution, so the same customer or supplier appears as several nodes and relationships are lost.
- Precomputing summaries that blend documents with different permissions, then showing them to every user.
- Budgeting for the first indexing run but not for keeping the graph in sync as sources change.
How Thinkia approaches it
We start from the questions, not the technique. With the people who own the content we collect real questions, classify them as local lookups, multi-step or corpus-wide, and agree what a correct answer and its evidence look like. That evaluation set decides the architecture: if most questions are local, a well-tuned hybrid retrieval with reranking is the right base, and a graph would add cost without adding answers.
Enterprise Knowledge AI provides a governed knowledge layer: ingestion that keeps source permissions, access control enforced at query time, answers linked to the document and passage that support them, and a full log of who asked what and what the system returned. Underneath, our RAG engine uses rerankers and, where they help, GraphRAG, agentic or multimodal patterns. When we add a graph, we scope it to a domain where relationships matter and treat entity resolution and permission handling as part of the design, not an afterthought.
Both options run on a model-agnostic base. With Synapse, retrieval can run on your own infrastructure and each task can be routed to the most suitable model, so the cost of extraction and summarisation stays visible and the decision can be revisited with data rather than opinions.
Thinkia products involved
- Enterprise Knowledge AIGoverned knowledge layer: source-grounded answers with citations and audit trail.
- SynapseGoverned agentic platform: agents, models, costs and data in one place.
Related AI solutions
Frequently asked questions
Is GraphRAG more accurate than vector RAG?
Not in general. It tends to help on questions that need connections across documents or a view of the whole corpus, and adds little on questions answered by a single passage. Measure both on your own questions before deciding.
Do I need a graph database to do GraphRAG?
Not necessarily. A graph can live in a dedicated graph database, in relational tables or alongside the vector index, depending on size and query patterns. What matters more is the quality of the extracted entities and relationships and how they are kept up to date.
Can an LLM build the knowledge graph automatically?
It can extract entities and relationships at scale, which is how many GraphRAG pipelines work, but the output needs checks: entity resolution, a defined set of relationship types and sampling by people who know the domain. Unchecked extraction errors become wrong links in answers.
We already have a knowledge graph or master data. Does that change the decision?
Yes. If curated entities and relationships already exist, for instance in master data or an existing graph, using them for retrieval is much cheaper than extracting a new graph from documents, and usually more reliable.
How does GraphRAG handle permissions?
With care. When an entity or a summary combines facts from documents with different access rules, a user may see something they could not read in the source. Keep provenance on every node and relationship and filter at query time, or limit the graph to content with uniform access.
What should we try before moving to GraphRAG?
Improve the basics first: chunking aligned with document structure, useful metadata, hybrid keyword and semantic search, reranking and query rewriting. Many gaps attributed to vector RAG close there, at a fraction of the cost of a graph.
Keep exploring
Related decisions
- RAG vs fine-tuning: which one does your enterprise use case need?
- RAG vs enterprise search: do your people need answers or documents?
- LLM evals vs human review: how should you assure the quality of enterprise AI?
- Fix the data first or start with AI now: which comes first?
Key terms
Thinkia articles
- Vector Databases for RAG: Powering Semantic Search for LLMs
- Agentic RAG: The Engine for High-Trust Enterprise Automation
- Fusion RAG: Boosting Accuracy and Relevance in Generative AI
- RAG Reranking: A New Path to Enterprise AI Accuracy and Speed