GraphRAG versus RAG: how to tell which one your question needs

Two different kinds of questions

RAG and GraphRAG are often pitched as competing options for the same job. They're not. They're better suited to two different kinds of questions, and the fastest way to pick wrong is to default to whichever one you already have running.

The test: can it be answered by finding similar text?

RAG is, at its core, a similarity search: given a question, find the passages that read most like an answer to it, and let a model summarize them. That works well when the answer is contained in a passage somewhere, more or less as written.

It works poorly when the answer isn't in any single passage, because it depends on how several entities relate to each other. "Which customers are exposed if this vendor fails" isn't answered by any one document. It's answered by tracing contracts to vendors to accounts, a multi-hop path through a relationship, not a sentence that happens to mention all three.

• Good fit for RAG: "What does our refund policy say about digital goods?" The answer is a passage.
• Good fit for GraphRAG: "Who on the account team has worked with both Acme and its parent company?" The answer is a path.
• Good fit for GraphRAG: "What's our total exposure if this supplier misses their next shipment?" The answer requires tracing dependencies, not finding a sentence.

When you need both

Plenty of real systems need both patterns behind the same interface: a support agent that retrieves policy text like RAG for most questions, but falls back to graph traversal when the question is actually about a relationship ("has this customer had this issue before, with this specific rep?"). Because both are built under the same manifest discipline and share the entity model where their sources overlap, adding graph traversal alongside retrieval doesn't mean standing up a second, disconnected system.