RAG vs Context Window

Both get information to the model, and they operate at different layers. The context window is what the model reads on one call, a fixed number of tokens. RAG (retrieval-augmented generation) is how you choose what goes into that window from a store too large to fit at once.

For a small corpus, pasting everything is right: under roughly 200k tokens, about 500 pages, you can skip retrieval and include it all. Past that size, two things break. Cost: every token in the window is re-read, and re-billed, on every call, so pasting the whole store multiplies the price of every turn. Recall: models read the middle of a long prompt worst, so a fact placed there is the one most often missed. A bigger window raises the point where this starts; it removes neither limit.