How much does it cost to build a RAG system in 2026?

RAG development cost by tier

RAG (retrieval-augmented generation) lets a language model answer from your documents and cite them. What you pay depends mostly on how many sources you connect and how far the answers must be trusted.

US agencies commonly publish $12,000 to $30,000 for a basic RAG build and $70,000 to $120,000 or more for enterprise RAG. Our ranges are lower mainly because our team works from Lahore, Pakistan, where costs are lower.

The scope behind our numbers is the same: ingestion, retrieval, evaluation, deployment and handover. If the system also needs to take actions, such as updating records or filing tickets, it becomes an agent. See our AI agent development cost guide.

What makes up the cost of a RAG system

The model call at the end is the easy part. Most of the budget goes into putting the right passage in front of the model and proving that it works.

Ingestion and parsing

PDFs with tables, scanned pages, slide decks and spreadsheets each need different handling. Clean HTML or Markdown is cheap to ingest. Scanned contracts with footnotes are not. Your document mix is the first thing we check in discovery.

Chunking

Documents are split into passages before they are embedded. Split too small and answers lose context. Split too large and search gets vague while token costs rise. Good chunking follows the document's own structure: sections, clauses, headings.

Hybrid search and re-ranking

Vector search finds passages with similar meaning. Keyword search finds exact terms such as part numbers, case names and clause references. Production systems usually run both, then re-rank the results before the model sees them.

Evaluation

We build a set of real questions with known answers and the passages that support them. Every change to chunking, search or prompts is scored against it. Without it, you can't tell whether a change helped or broke something.

Permissions

If different users may see different documents, retrieval must filter by access rights before anything reaches the model. Syncing those rights from the source systems is a large part of the gap between the production and enterprise tiers.

Interface and citations

Users trust answers they can check. Linking each answer to the exact passage takes work in both the retrieval layer and the UI. We include it in every production build.

Monthly RAG infrastructure costs

Running a RAG system has four main lines: the language model, embeddings, the vector database and hosting. For typical workloads the total lands between roughly $100 and $2,000 a month, paid to the providers at cost.

A worked example

Take 50,000 pages at about 500 tokens a page, or 25 million tokens. Embedding them once costs about $0.50 with OpenAI's small embedding model, or about $3.25 with the large one. Embeddings are rarely the problem.

Answering is where the money goes. Say staff ask 1,000 questions a day, about 30,000 a month. Each question sends around 6,000 tokens of retrieved context and gets a 500-token answer.

Better retrieval lowers this bill. Sending 3 sharp passages instead of 15 loose ones cuts input tokens and tends to improve answers. Prompt caching and batch processing lower it further for repeated or offline work.

Costs buyers often miss

• Re-indexing. Switching embedding models means embedding the whole corpus again and re-running your evaluation set. The tokens are cheap; the testing time is not.
• Document sync. Sources change every day. Jobs that pick up edits and remove deleted files need hosting and monitoring of their own.
• Model changes. Providers retire models and change prices. Each switch needs a test run before it goes live.
• Re-ranking and parsing services. Paid re-rankers and document parsing APIs add per-call fees. Open-source options trade those fees for hosting.

Basic vs production RAG

A basic RAG demo can be built in days from a tutorial. It works on the ten documents you tried it on. Production RAG has to work on the ten thousand you didn't.

The gap is mostly testing and data handling, not the model. That is why a demo can look almost done and still need weeks of work.

Build vs buy: when custom RAG is worth it

Off-the-shelf tools are often enough. Buy when:

• Your documents already live in a tool with built-in AI search, and its answers are good enough.
• A small team needs it for internal reference, and a wrong answer costs little.
• Your documents are clean, mostly text, and don't need per-user permissions.

Build custom when:

• Answers must cite the exact passage, as in legal, compliance or technical support work.
• Data spans several systems or must respect per-user permissions.
• Data can't go to a third-party SaaS tool, or must stay on your own servers.
• Search needs domain logic, such as part numbers, case citations or product filters.
• The feature is part of a product you sell, so you need to own its quality and cost.

A sensible path is to try an off-the-shelf tool on your real documents for a few weeks. If it fails on accuracy, permissions or privacy, you now have a clear spec for what to build.

RAG vs fine-tuning cost

Fine-tuning changes how a model writes. RAG changes what it knows at answer time. Most teams asking about fine-tuning for company knowledge need RAG.

For company knowledge, RAG is usually cheaper to build and much cheaper to keep current. Fine-tuning can make sense later, on top of RAG. One example is teaching a small model your output format so you can move to a cheaper tier.

Self-hosted RAG for sensitive data

Legal, health, finance and security teams often can't send documents to a third-party API. RAG can run fully inside your environment: open-weight models, a self-hosted vector database and your own servers.

This changes the cost in three ways.

• Build cost rises. Model serving, GPU setup and monitoring add work. It often pushes a project toward the enterprise tier.
• Token fees go away, GPU bills arrive. You pay for GPUs whether they are busy or idle. One always-on GPU costs roughly $430 to $2,900 a month on a GPU cloud at on-demand rates, depending on the card.
• Quality needs checking. Test open-weight models on your own evaluation set before you commit to them.

A middle path is a private deployment of a hosted model inside your own cloud account. Data stays under your contracts, and you don't run GPUs yourself. We cover the security side on our AI security page.

Real examples from our work

midpage.ai: legal research with exact citations

We built midpage.ai, an AI legal research product, end to end. It gives retrieval-augmented answers, and every answer links to the exact passage in the source document.

In legal work, an answer without a source can't be used. Passage-level citations depend on careful parsing and retrieval, which is where this kind of build spends its effort. See the project.

A self-hosted assistant for a US distributor

For a US B2B distributor of low-voltage and security products, we built a self-hosted AI shopping assistant. Open-weight models run on the client's own servers with Ollama and vLLM, so customer data never leaves their environment. Typesense powers hybrid product search.

One lesson applies to any RAG budget: keep product search and RAG document chunks in separate search collections. Merging them inflated facets, caused redundant embedding calls and created pagination loops. See the project.

How we price RAG projects

We start with a 1–2 week discovery sprint at $1,500–$3,000. We take a sample of your documents, test parsing and retrieval on them and write the evaluation questions with you. It ends with a fixed quote.

The build is paid in milestones. Infrastructure is billed to you by the providers or passed through at cost. If you need help after launch, a dedicated engineer costs $35–$45 an hour.

Tell us what documents you have and who needs answers from them, then book a call. More on our LLM and RAG development service.