RAG engineer job description: what the role does and a template you can post

What does a RAG engineer do?

A RAG engineer designs, builds and maintains systems that answer questions from an organisation's own content by retrieving relevant passages and generating grounded responses with a language model. The role sits between search engineering, data engineering and LLM application development: it owns document ingestion, chunking and embedding, the vector and keyword indexes, retrieval and reranking, the generation prompt with citations and refusal rules, and the evaluation and monitoring that prove the system is accurate.

The output is usually an internal assistant, a support bot, a document search product or a knowledge layer that other AI agents call. The retrieval idea comes from Lewis et al. (2020), but the production job in 2026 is mostly engineering: data quality, search relevance, access control, cost and latency. If you are unsure whether the role should be retrieval-focused or model-focused, RAG vs fine-tuning explains the split.

What are the responsibilities?

Use eight to ten of these, worded for your system.

• Build and maintain ingestion pipelines that parse PDFs, web pages, tickets, wikis and structured records into clean, chunked, metadata-rich documents.
• Select and evaluate embedding models and chunking strategies against a labelled question set, not by default settings.
• Design hybrid retrieval combining vector search with BM25 keyword search, metadata filters and a reranker where evaluation shows a gain.
• Implement query rewriting and conversation-aware reformulation for multi-turn use.
• Write generation prompts that answer only from retrieved context, cite sources and refuse when the documents do not cover the question.
• Enforce document-level permissions at retrieval time, synced from source systems.
• Keep indexes fresh through incremental re-indexing, document versioning and planned re-embedding when models change.
• Own the evaluation suite: retrieval recall and precision, faithfulness, answer relevance, latency and cost per query, with regression runs on every change.
• Operate the system in production: monitoring, failure sampling, feedback loops and on-call for retrieval incidents.

Which skills are required?

• Shipped at least one RAG system to real users and can describe its corpus, metrics and failures.
• Strong Python and production habits: testing, observability, CI.
• Hands-on with a vector database such as pgvector, Qdrant, Weaviate or Pinecone, including filtering and index tuning.
• Working knowledge of information retrieval: BM25, dense retrieval, hybrid fusion, cross-encoder reranking, evaluation at k.
• Experience with hosted LLM APIs or self-hosted models, including structured output, streaming and prompt caching.
• Document processing: PDF and HTML parsing, table extraction, OCR where needed, chunking by structure.
• Evaluation: building golden sets, RAGAS-style metrics, LLM-as-judge calibrated against human review.
• Security awareness: access control in retrieval, prompt injection through documents, handling of sensitive data.

What is nice to have?

• GraphRAG or knowledge-graph experience for corpus-wide questions.
• Fine-tuning embedding models or rerankers on domain data.
• Experience serving open-weight models with vLLM or similar, and the cost trade-offs versus hosted APIs.
• Agent frameworks that call retrieval as a tool (LangGraph, the OpenAI Agents SDK, MCP servers).
• Domain experience in your industry, for example legal, healthcare, fintech or customer support.

Copy-paste job description template

Replace the bracketed parts and keep the whole posting under 600 words.

About the role

We are hiring a [Senior] RAG Engineer to build and run the retrieval layer behind [product or internal assistant]. You will own how [corpus: for example 200,000 support articles and contracts] is ingested, indexed and retrieved, and how answers are generated, cited and evaluated. The system serves [users and volume] and must meet [quality and latency targets]. You will report to [role].

Responsibilities

Design and maintain ingestion and chunking pipelines. Choose and evaluate embedding models, hybrid retrieval and reranking. Build grounded generation with citations and refusal behaviour. Enforce permissions at retrieval time. Own the evaluation suite and production monitoring. Scope new corpora with product and data owners.

Requirements

[3+] years of software engineering in Python, with at least one RAG or search system shipped to production. Hands-on experience with a vector database and with information retrieval concepts (BM25, dense retrieval, reranking, recall at k). Experience with LLM APIs or self-hosted models. Evidence of building evaluation sets and using them to drive decisions.

Nice to have

GraphRAG or knowledge graphs. Fine-tuning embeddings or rerankers. vLLM or other self-hosted serving. Agent frameworks that use retrieval as a tool. Experience in [industry].

How we work

Remote, [time zone overlap requirement]. [Contract or full-time], [hours]. [Interview process: a screen, a technical round, a design exercise and a production round, within ten days]. [Compensation range and benefits, or rate range for contractors].

How does the role change by seniority?

• Junior (0 to 2 years): implements components under direction: parsers, chunkers, embedding jobs, evaluation scripts. Needs a senior to set the retrieval design and quality targets.
• Mid-level (2 to 5 years): owns a pipeline end to end for one corpus, tunes chunking and hybrid search against an evaluation set, handles re-indexing and monitoring. Still needs help with permissions, cost modelling and multi-corpus architecture.
• Senior (5+ years): designs the retrieval architecture, sets the evaluation strategy, makes the long-context versus RAG and GraphRAG calls, owns permissions and production operations, and can estimate and cut cost per query.
• Lead or staff: sets retrieval standards across teams, chooses the platform (vector store, serving, evaluation tooling), works with security and data governance, and turns business quality targets into engineering metrics.

What do RAG engineers earn?

"RAG engineer" is rarely a standalone title on salary sites, so benchmark against AI engineer and machine learning engineer data and expect a premium for proven production retrieval experience. For US full-time roles, Levels.fyi (updated 17 October 2026) reports a median total compensation of $280,000 for machine learning engineers, with a 25th to 75th percentile range of $200,000 to $385,000; that data skews to large technology companies and includes equity. Glassdoor's broader average base salary for AI engineers is about $140,678, as quoted in KORE1's AI engineer salary guide (5 August 2026), which also gives 2026 base ranges of $140,000 to $210,000 for mid-level and $180,000 to $280,000 for senior engineers.

For contractors, Arc.dev states that AI developers hired through its platform "typically charge between $60-100+/hour", and Lemon.io's Q3 2026 rate report puts strong-senior AI engineers at $74 to $89 an hour with a median of $81. Hyparz publishes indicative hourly bands for AI engineers that track the rates Arc.dev publishes; the quote depends on specialism, seniority and engagement length. The loaded cost of a full-time hire is worked through in what it costs to hire AI engineers.

What are the common mistakes in RAG job descriptions?

• Listing frameworks instead of outcomes. "LangChain, LlamaIndex, Pinecone" attracts tutorial builders; "lift retrieval recall at 10 above 90 percent on our evaluation set" attracts engineers.
• Treating it as a prompt engineering role. Most RAG quality lives in ingestion and retrieval; a posting that leads with prompts underweights the hard part.
• Omitting permissions and data sensitivity. If users have different access rights, say so; it changes the architecture and the candidate pool.
• Asking for everything: MLOps, fine-tuning, agents, front-end and RAG in one person. Split the posting or accept a longer search.
• No corpus description. Document types, volume, languages and how often content changes tell candidates what the job really is.

Where should you post it, and how do you hire faster?

Post on the general boards for reach, but expect the signal to come from AI-specific communities, open-source RAG projects and referrals from engineers who have shipped retrieval. Screen on the RAG engineer interview questions and keep the loop under ten days; how to hire AI engineers covers the process.

If you need the role filled in weeks rather than months, hire through a vetted marketplace. Hyparz matches companies with pre-vetted remote RAG developers, typically within days; about 5% of AI-role applicants pass vetting. Every engagement starts with a one-week risk-free trial where you keep any work delivered, with a dedicated account manager, automated timesheets and source-code protection. If you want the system built rather than the seat filled, see RAG implementation services. Contractor vs full-time helps decide which to post.