How many documents can a RAG system handle?
A home-made RAG tops out at a few thousand documents per agent, fewer if they look alike. An agentic RAG that filters first handles over 100,000.
How many documents can a RAG system handle? It depends on its architecture. A simple RAG, the kind teams build in-house with a vector database and a search over the whole corpus, stays reliable up to a few hundred, at most a few thousand documents per agent. Beyond that, storage holds up fine; search precision is what gives way. And the threshold drops further when documents look alike: a few hundred contracts built on the same template for different clients are enough to throw a simple RAG off. An agentic RAG, which filters the corpus before running the vector search, absorbs more than 100,000 documents. At several of our clients, ATG agents run in production on more than 100,000 documents, or over a million pages.
Both figures are out there, and both are true. They describe two different systems.
Where does the "few hundred documents" figure come from?
It has two origins, and neither says much about RAG technology itself.
The first is the caps of consumer tools. Search for "how many documents can I give an AI" and you land on the limits of products everyone knows. As of October 2026, NotebookLM accepts 50 sources per notebook on the free tier and a few hundred on paid plans. ChatGPT projects and custom GPTs accept a few dozen files depending on the plan. These are product decisions, sized for one person working on one file.
The second is feedback from home-made RAGs. The classic tutorial (split documents, compute embeddings, retrieve the ten closest chunks, hand them to the model) produces an impressive demo on fifty documents. Then teams add files, and quality drops without anything visibly breaking. Hence an empirical consensus: "beyond a few thousand documents, it doesn't work very well anymore."
That consensus is accurate, for that architecture.
Why a simple RAG loses precision as the corpus grows
A simple RAG works like this: the question is turned into a vector, the system searches the entire corpus for the chunks with the closest vectors, keeps a fixed number of them (often between 5 and 20), and the model answers from those chunks only.
On 50 documents, this is trivial. On a million pages, the same mechanism has to pick ten chunks out of more than a million. Every document added brings its share of passages that are roughly about the question's topic without answering it, and they end up taking the ten slots.
Old versions make it worse. The bigger the base, the more drafts and copies it holds. The 2022 procedure and the 2026 one look very much alike, and the system has no reason to prefer the right one.
So do proper nouns. A client name, a product reference or a project code is often the only word that matters in a question, and it is the one an embedding represents worst: these are keys with no general meaning. We covered this in our article on hybrid search.
The result is insidious: the assistant keeps answering confidently, citing sources. Just less and less often the right ones.
Look-alike documents: the worst case for a simple RAG
To my mind, this is the most misunderstood part of the topic. People talk about volume and forget about similarity. Yet two corpora of the same size can behave in opposite ways.
Take 5,000 heterogeneous documents: company handbook, product sheets, quality procedures, legal notes. A question about leave naturally lands on the HR documents, because their meaning stands apart from everything else. The simple RAG copes fairly well.
Now take 500 client contracts drafted on the same template. A salesperson asks: "What is the termination notice period in the Durand contract?" All 500 contracts have a termination clause, almost word for word. To a vector search, these are 500 equivalent passages. What sets them apart fits in one word, Durand, and that is precisely the word it captures worst. The ten chunks it keeps will come from ten random clients, and reranking won't fix that: it reorders the chunks it is given, and the right one is probably not among them.
The more documents look alike, the more a simple RAG malfunctions, and the sooner. The "few thousand" ceiling then drops to a few hundred.
This is far from an edge case. Contracts, audit reports, due diligence files, site visit reports, tender responses: this is the daily work of consulting, audit and law firms, M&A teams, insurers and engineering firms. These businesses produce the same kind of deliverable for dozens or hundreds of clients, and those are the documents that matter most to them.
The fix is to treat the client name as a filter, applied before any search for meaning, instead of a word drowned in a vector. That is the principle of agentic RAG.
What research says about scaling
A study published in July 2026, BM25 Wins at Scale, measured several RAG architectures on a corpus grown step by step over a 450-fold range, with the same questions from start to finish. Two findings matter here.
Vector search alone stays fast, but it loses accuracy against keyword search as the corpus grows. These are the "roughly on topic" passages described above.
An agent that explores files without an index, like someone opening folders one by one, excels on a small corpus and collapses at scale: it uses 39 times more tokens and ends up nearly 20 points behind. The authors conclude that agentic reasoning works best after a ranked first search.
So you need both: indexes that find things fast, and an agent that reasons about what they find.
Agentic RAG: filter before you search
An archivist asked for the notice period in the Durand contract does not reread 500 contracts. They go to the Clients folder, open the Durand subfolder, take the latest version of the contract and look for the clause.
An agentic RAG does the same. Before any vector search, the agent narrows the scope.
- It works out what the question points to: a specific document (the contract), a client (Durand), a concept (the termination notice).
- It queries the document catalog. Sources, folders, paths, dates, statuses: the catalog answers deterministically, with no semantic approximation. The scope shrinks from 100,000 documents to a handful.
- It searches within that subset only, combining semantic and keyword search, then sorts the chunks through reranking. The 499 other termination clauses are no longer even candidates.
- It checks. If the chunks are not enough, it reads the whole document, or goes back with what it just learned.
The vector search no longer picks ten chunks out of a million, but out of a few hundred, where it is highly reliable. The total size of the corpus stops being the limiting factor, since the fine-grained search never runs over the whole corpus.
Diagram comparing two RAG architectures on 100,000 documents. On the left, the simple RAG runs a vector search over the entire corpus and keeps ten chunks, many of them false neighbors from other documents. On the right, the agentic RAG first queries the document catalog to narrow the scope to about forty documents, then runs a hybrid search with reranking within that subset and produces a sourced answer.
Simple RAG vs agentic RAG: the comparison
| Simple RAG | Agentic RAG with filtering | |
|---|---|---|
| Reliable volume per agent | A few hundred to a few thousand documents | Over 100,000 documents, over a million pages |
| Search scope | The whole corpus, for every question | A subset chosen by the agent |
| Same-type documents for different clients | The worst case: breaks down from a few hundred | The client becomes a filter, applied before the search |
| Proper nouns and references | Poorly retrieved | Retrieved through keywords and the catalog |
| Inventory questions ("how many contracts…") | Approximate or wrong answer | Exact answer, from the catalog |
| Competing versions | Mixed up | Resolved by date, status or folder |
| Response time | A few seconds | A few more seconds, depending on round trips |
Agentic RAG comes at a cost. Each step uses tokens and time, and an agent without guardrails can go round in circles. You need to cap the number of round trips and trace every source, otherwise the system gets slow and its answers unverifiable.
Splitting into several agents: a false fix
When a simple RAG saturates, the reflex is to create one agent per department, then per topic, or even per client, to keep each corpus under the few-thousand mark. That is where the idea of a limit "per agent" comes from.
It works, but the burden shifts to the user, who now has to know which agent to ask. The question "are we meeting our commitments to Durand?" touches contracts, support and billing at once, and three siloed agents cannot answer it. As for one agent per client, with 300 clients it quickly becomes unmanageable.
There are good reasons to separate agents, and they come down to access rights: an HR agent has no business reading client contracts. In ATG, that scope is set through the knowledge categories assigned to each agent.
What it takes to run 100,000 documents in production
The agentic architecture lifts the precision ceiling. Other issues show up at this scale, and it is better to know them upfront.
Ingestion becomes an industrial project. A million pages means scanned PDFs, spreadsheets, slide decks, and an OCR bill that is far from trivial. We costed the gap between an API and an open-weight model on a million pages in our Mistral OCR vs PaddleOCR comparison: from $4,000 to under $200.
Filtering is only as good as the metadata. The agent relies on what the catalog knows: source, folder, date, author, status. A coherent SharePoint structure serves it far better than a 40,000-file "Misc" folder.
Sync must be incremental. You don't reindex a million pages every night, only the documents that changed.
Quality is measured on real questions. A 100,000-document RAG is not validated with ten questions in a steering committee. You need hundreds of representative questions, including tricky ones: one specific document among thousands of similar ones, an internal reference, an inventory question.
How to test a RAG solution's capacity before buying
A vendor claiming "unlimited" tells you nothing. Four tests are enough to tell solutions apart, and we would put anyone through them, ourselves included.
- Ask for the production volume, not the demo one: how many documents and pages at a real client, and across how many agents.
- Look for a needle among look-alike documents. Pick a document buried among hundreds of documents of the same type (one client's contract among all contracts, one audit report among all reports) and ask a question whose answer is only in that one. It is the most telling test.
- Ask a question with a proper noun or an internal reference, which semantic search alone retrieves poorly.
- Ask an inventory question, such as "how many contracts were signed in 2025?" An approximate answer gives away a search with no catalog.
The Ask This Guy approach
At Ask This Guy, the agent drives the search. It has several complementary tools: the document catalog to locate and count documents, relevant-document search to identify a file, hybrid search (semantic and keyword) to find the passage, reranking to keep the best chunks, and full-document reading when a chunk is not enough.
Our clients connect more than 100,000 documents this way, over a million pages, to a single agent, without splitting their base to stay under a threshold. Answers cite their sources, whatever the size of the corpus.
If your project is hitting a volume wall, take a look at our turnkey enterprise RAG, or book a demo to test it on your own documents.
Frequently asked questions about how many documents a RAG can handle
How many documents can a RAG system handle?
It depends on its architecture. A simple RAG, which runs a vector search over the whole corpus for every question, stays reliable up to a few hundred or a few thousand documents per agent, and fewer still when documents look alike. An agentic RAG, which filters the corpus before searching, handles more than 100,000 documents, or over a million pages: that is the volume Ask This Guy agents handle in production. The limit comes from search precision; storage keeps up without difficulty.
Why does a RAG lose precision as the number of documents grows?
Because a simple RAG picks a fixed number of chunks, often ten, out of the entire corpus. The bigger the corpus, the more passages it holds that are close in meaning to the question without being the right answer: similar clauses from different contracts, old versions, copies. Proper nouns and internal references, poorly captured by embeddings, make it worse. Eventually the right answer is no longer among the chunks retrieved.
Why does a RAG perform poorly on similar documents?
Because vector search compares the meaning of passages, and documents of the same type (contracts on the same template, audit reports, client files) almost all share the same meaning. What sets them apart, a client name, a date, an amount, is precisely what an embedding captures worst. A simple RAG then brings back passages from other clients. The fix is to filter first on the client, folder or source, then search within that subset only.
What is filtering before vector search in a RAG?
It is the step where the agent narrows the scope before searching for meaning. It queries the document catalog (source, folder, date, status, client or project name) to isolate the few relevant documents, then runs semantic and keyword search within that subset only. The agent decides on this filter from the question, without the user having to specify it.
Can a RAG handle a million pages?
Yes, as long as it does not search the million pages for every question. An agentic RAG filters the corpus first, then searches a reduced subset, which keeps the search precise whatever the total size. A million pages mostly raises ingestion questions: OCR cost, incremental sync of modified documents, metadata quality.
Should you create several agents to spread a large number of documents?
Not for volume reasons, if the architecture filters before searching. Spreading documents across several agents forces users to know which one to ask, and rules out questions that span several domains. The right criterion for separating agents is access rights: an HR agent has no business reading client contracts.
Conclusion
"A few hundred, at most a few thousand documents": the answer you find online is right for a simple RAG, and it explains why so many in-house projects plateau after the POC. It describes one specific architecture, the one that searches everywhere for every question.
The number of documents is only half the problem, too. At equal volume, look-alike documents make a simple RAG fail much sooner.
An agent that first knows where to look, before searching for what, removes the ceiling, even across hundreds of look-alike documents. You go from a few thousand documents to over 100,000, and a company can finally connect its entire document base rather than a carefully chosen sample.
