Ask a standard RAG system “What are the main themes across these 500 clinical trial reports?” and it will retrieve the five chunks most semantically similar to the word “themes.” It will not synthesize a thematic analysis. It cannot. It was designed to find a needle in a haystack, not to describe the haystack.

This is the fundamental limitation that GraphRAG was built to solve. And the solution — building a knowledge graph from unstructured text, clustering it hierarchically, and pre-computing community summaries — is one of the most consequential architectural shifts in retrieval-augmented generation since the concept was invented.

The problem vector RAG cannot solve

Traditional RAG chunks documents into vectors and retrieves the top-k most similar chunks based on a query embedding. It works well when the answer lives in a specific chunk: “What was Q3 revenue?” or “What does Section 4.2 of the SOP say?”

It fails on three categories of queries:

Global sensemaking. “What are the primary recurring themes in these 500 reports?” requires synthesizing information scattered across the entire corpus. Vector search retrieves isolated snippets, not synthesized concepts.

Multi-hop reasoning. “How is Person A connected to Organization B through intermediaries?” requires traversing implicit relationships across documents that share no direct keyword overlap. Vector similarity cannot follow relationship chains.

Context fragmentation. Complex documents have hierarchical structure, relational context, and distributed narratives. Chunking destroys all of this. The retrieved chunks are semantically similar but structurally disconnected.

GraphRAG addresses all three by introducing a graph-mediated retrieval layer between the raw text and the LLM.

The architecture: indexing and query

Microsoft Research published the reference GraphRAG framework in 2024. It operates in two phases: a heavy offline indexing phase and a lightweight runtime query phase.

Indexing pipeline

The indexing pipeline transforms unstructured text into a structured knowledge base through six stages:

1. Document chunking. Source documents are split into TextUnits — typically 1,200 tokens with 100-token overlap. These serve as fine-grained, citable references for all downstream outputs.

2. Entity and relationship extraction. An LLM (GPT-4o by default) scans each chunk using structured prompting to extract typed entities (Person, Organization, Location, Event), their relationships (directed edges with descriptions), and key claims. This is the most expensive step — one or two LLM passes per chunk.

3. Graph construction and deduplication. Extracted entities are merged, resolving synonymous names (“Sam Altman” and “S. Altman”) into canonical nodes with aggregated edge weights and combined descriptions.

4. Hierarchical community detection. The Leiden algorithm partitions the graph into nested, modular subgraphs at multiple granularities. Level 0 represents the highest-level themes, Level 1 more granular topics, and so on. This is a pure graph algorithm — no LLM calls.

5. Community summarization. An LLM generates narrative summary reports for each detected community at each hierarchical level, capturing the collective context of all nodes within that subgraph. This is the architectural breakthrough — pre-computing “answers” to potential global questions.

6. Embedding. TextUnits, entity descriptions, and community reports are embedded for vector retrieval.

Source Documents
      │
      ▼
┌─────────────┐
│  Text Chunk  │  (1,200 tokens, 100 overlap)
│  Segmentation│
└──────┬──────┘
       │
       ▼
┌─────────────┐
│   Entity &   │  LLM extraction per chunk
│ Relationship │  (entities, types, edges, claims)
│  Extraction  │
└──────┬──────┘
       │
       ▼
┌─────────────┐
│   Graph      │  Merge synonyms, canonicalize
│ Construction │  nodes, aggregate edge weights
│ & Dedup      │
└──────┬──────┘
       │
       ▼
┌─────────────┐
│  Community   │  Leiden algorithm — hierarchical
│  Detection   │  clustering (no LLM)
└──────┬──────┘
       │
       ▼
┌─────────────┐
│  Community   │  LLM generates narrative
│ Summarization│  summaries per community
└──────┬──────┘
       │
       ▼
┌─────────────┐
│  Embedding   │  Vector store for retrieval
└─────────────┘

Query modes

At query time, GraphRAG branches based on the nature of the question. Four search strategies are available:

Local Search — entity-centric. Identifies key entities in the query via vector search, traverses neighboring relationships and incident edges, pulls the raw source chunks associated with those nodes, and sends that concentrated context to the generator model. Best for “Tell me about Entity X” or specific factual queries.

Global Search — holistic corpus reasoning. Bypasses raw chunks entirely. Samples community summaries across different hierarchical layers, uses a map-reduce step to score each summary’s relevance to the prompt, and synthesizes a global answer from the top-scoring community reports. Best for “Summarize the main themes” or “What are the systemic vulnerabilities?”

DRIFT Search (Dynamic Reasoning with Iterative Tree expansion) — a hybrid introduced in 2024. Expands the query into follow-up questions using community insights to refine local search. Best for “How does X relate to broader themes?” — queries that need both entity-level detail and corpus-level context.

Basic Search — traditional vector similarity fallback for simple factoid queries where graph traversal is unnecessary.

Search Mode Best For Context Source Token Cost
Local Specific entities, direct facts Node attributes, 1-2 hop edges, source chunks Low
Global Holistic summaries, themes Pre-computed community summaries High (map-reduce)
DRIFT Entity + broader context Community insights + local traversal Medium
Basic Simple factoid lookup Vector similarity on raw chunks Minimal

The community summaries innovation

This is GraphRAG’s key architectural contribution. Instead of retrieving raw text chunks for global questions, it retrieves pre-synthesized summaries of subgraph communities.

A corpus of 10,000 chunks might reduce to 50 community summaries at the top level. Each summary already encodes distributed knowledge — “Customers in Region A frequently mention pricing concerns, while Region B mentions delivery delays.” The LLM at query time performs reduction (synthesizing summaries) rather than extraction plus synthesis (finding relevant bits then making sense of them).

The hierarchical structure allows dynamic precision/recall trade-offs. Zoom out for broad trends (root-level communities), zoom in for specific domains (leaf-level communities). Microsoft’s evaluation showed 70-80% win rates over naive RAG on comprehensiveness and diversity metrics for global questions, with root-level summaries achieving competitive performance at just 2-3% of the token cost of full source text.

The cost problem

GraphRAG’s power comes with a price tag that drives most architecture decisions.

Indexing cost. Full Microsoft GraphRAG indexing costs approximately $50 per million tokens with GPT-4, compared to approximately $0.50 for vector embedding. That is a 100x difference at index time. A 1,000-page corpus can cost $50-200 to index with commercial LLMs.

Query cost. Global search is the expensive mode. It requires sampling community summaries across hierarchical layers, with worst-case token usage reaching approximately 610,000 tokens per query across hundreds of LLM calls. LightRAG, by comparison, uses fewer than 100 tokens per query in a single call — a 6,000x differential.

Latency. Graph traversal plus multiple LLM calls increases response time. In a biomedical benchmark, GraphRAG averaged 15 seconds per query versus 3.35 seconds for vector RAG.

Phase Vector RAG Microsoft GraphRAG LightRAG LazyGraphRAG
Indexing (1M tokens) ~$0.50 ~$50 ~$6-7 0.1% of full GraphRAG
Query (global) N/A ~610K tokens <100 tokens Comparable to vector RAG
Query (local) ~4-8K tokens Similar Similar Similar

These numbers explain why the field has rapidly diversified into cost-optimized variants.

The variant landscape

Microsoft’s original GraphRAG spawned an ecosystem of alternatives, each trading graph richness for efficiency.

LazyGraphRAG

Released by Microsoft Research in late 2024, LazyGraphRAG radically defers all LLM use to query time. Instead of LLM-based entity extraction during indexing, it uses NLP noun-phrase extraction to identify concepts and co-occurrences — making indexing cost identical to vector RAG (0.1% of full GraphRAG).

At query time, LazyGraphRAG uses a tunable relevance test budget to control cost-quality tradeoff. The key insight: most community summaries in full GraphRAG are never queried, so pre-computation is wasteful. LazyGraphRAG introduces best-first chunk ranking with breadth-first relevance assessment and iterative deepening, achieving comparable quality to global search at 4% of the cost.

LightRAG

From the University of Hong Kong (EMNLP 2025), LightRAG is the most cited lightweight alternative. Its core innovation is dual-level retrieval: low-level keywords for specific entity names and high-level keywords for macroscopic themes. It supports incremental graph-union updates without full re-indexing — a critical feature for dynamic corpora.

Independent benchmarks show LightRAG indexes 3-5x faster and costs 50-70% less than full GraphRAG with comparable quality. The caveat: “LightRAG is lighter than global GraphRAG, not light in absolute terms” — some runs still require approximately 4,900 seconds and 10^4 token prompts.

HippoRAG

Neurobiologically inspired. HippoRAG models the LLM encoder as the neocortex and the knowledge graph plus Personalized PageRank as the hippocampus. It uses PPR seeded from query entities to spread activation through the graph, outperforming state-of-the-art by up to 20% on multi-hop QA while being 10-30x cheaper and 6-13x faster than iterative retrieval methods.

Other variants

Variant Key Idea Differentiator
Fast GraphRAG Lightweight heuristics, embedding-based clustering, PageRank retrieval 10x cost reduction, 27x faster
Nano-GraphRAG Minimal ~1,100-line Python implementation Educational, Ollama-native
PropRAG Belief propagation through text chunk graph No explicit KG extraction
RAPTOR Recursive clustering into tree structure Hierarchical without explicit graph
LinearRAG Relation-free construction (ICLR 2026) Near-zero indexing cost

The spectrum is clear: from full LLM extraction (most accurate, most expensive) through NLP-only extraction (0.1% cost) to relation-free approaches (near-zero cost). The field is converging toward lazy, NLP-based construction with LLM involvement deferred to query time.

Evaluation: when does graph structure actually help?

This is the question that matters, and the answer is nuanced.

Microsoft’s original evaluation

GraphRAG achieved 70-80% win rates over naive RAG on comprehensiveness and diversity for global sensemaking questions. Intermediate- and low-level community summaries outperformed source text summarization at 20-70% token cost. Root-level summaries were competitive at 2-3% token use.

What was not proved: superiority for local fact retrieval, cost-effectiveness at scale, or generalizability beyond query-focused summarization.

GraphRAG-Bench (ICLR 2026)

The first comprehensive benchmark explicitly designed to answer “When to use Graphs in RAG?” — accepted at ICLR 2026. It tests tasks of increasing difficulty across two domains (novel and medical): fact retrieval, complex reasoning, contextual summarization, and creative generation.

The critical finding: GraphRAG frequently underperforms vanilla RAG on many real-world tasks. Performance varies dramatically by task type — fact retrieval often favors vector RAG, while contextual summarization and creative generation favor graph methods. The benefit is conditional, not universal.

Independent benchmarks

Study Vector RAG GraphRAG Delta
AWS ML Blog (custom dataset) 50.83% correct 80% correct, 90% acceptable +29pp
Biomedical (500 PubMed) 774 tokens, 3.35s 594 tokens, 15.01s -23% tokens, +4.5x latency
EntityNet (entity risk) 0.355 F1 0.681 F1 +92%

Three failure modes emerge from independent evaluation: extraction errors (the LLM misidentifies entities or relationships), community noise (poorly formed communities produce misleading summaries), and poor performance on simple lookup queries where vector RAG is faster and cheaper.

When to use GraphRAG — and when not to

Use GraphRAG when:

  • Queries require cross-document entity linkage (“Are there individuals accused in multiple cases?”)
  • The primary questions are global sensemaking (“What are the main themes across all our customer feedback?”)
  • Multi-hop reasoning matters (“How does Drug A relate to Adverse Event C through intermediate mechanisms?”)
  • Compliance and auditability require deterministic source tracing back to specific graph nodes
  • The corpus is large (1M+ tokens) and the questions are holistic

Do not use GraphRAG when:

  • Simple factoid lookup over small corpora where vector RAG achieves comparable accuracy at 1/100th the cost
  • Latency requirements are under 1 second
  • The data is highly dynamic and requires real-time updates (unless using LightRAG incremental or LazyGraphRAG)
  • Extraction quality is low (noisy OCR, highly technical jargon without tuned prompts)
  • The corpus is small enough for the LLM to ingest in context

The decision framework:

IF query == global_sensemaking (themes, summarize corpus):
    → GraphRAG or LazyGraphRAG (70-80% win rate)

ELIF query == multi_hop_entity_linking AND corpus > 1M tokens:
    → LightRAG (dual-level) or HippoRAG (PPR)

ELIF query == local_factoid AND latency < 2s:
    → Vector RAG

ELIF budget == severe:
    → LazyGraphRAG (0.1% indexing) or FastGraphRAG

ELIF data == highly dynamic:
    → LightRAG (incremental updates)

ELSE:
    → Hybrid: Vector for local + Graph for global

The pragmatic path forward

GraphRAG is not a universal replacement for vector RAG. It is a complementary paradigm that excels when the question requires synthesizing dispersed information, understanding entity relationships, or summarizing an entire corpus.

The cost differential is real and shrinking. Full Microsoft GraphRAG indexing at $50 per million tokens was prohibitive in 2024. LazyGraphRAG brought indexing cost to parity with vector RAG. LightRAG cut retrieval tokens by 6,000x. HippoRAG made multi-hop retrieval 10-30x cheaper. LinearRAG (ICLR 2026) pushed indexing cost to near zero.

For most organizations in 2026, the right architecture is hybrid: vector RAG for low-latency local lookups, LightRAG or LazyGraphRAG for cost-sensitive graph reasoning, and full Microsoft GraphRAG with DRIFT and dynamic community selection when comprehensiveness on global sensemaking queries justifies the upfront indexing investment.

The graph is not the retrieval method. The graph is the structure that makes retrieval intelligent. Build accordingly.

Sources: Microsoft Research, arXiv (2404.16130, 2410.05779, 2405.14831), Neo4j Developer Blog, Weaviate, GraphRAG-Bench (ICLR 2026), LearnOpenCV, GitHub (HKUDS/LightRAG, neal-wu/from-rag-to-llm)