LLM engineer job description: responsibilities, skills and a template you can post today

What does an LLM engineer do?

An LLM engineer, also called an LLM application engineer or LLM developer, turns a language model into a reliable product feature. The role sits between backend engineering and applied machine learning: the engineer writes application code, designs how the model is prompted and constrained, decides what tools and data it can reach, measures its output quality with evaluation sets, and keeps latency, cost and safety within budget once real users arrive.

The role differs from a machine learning engineer, who trains and deploys models, and from a prompt engineer, who tunes prompts without owning the system around them. Most LLM engineers work with hosted model APIs and, where volume or data control demand it, with open-weight models served through vLLM or similar.

What are the responsibilities of an LLM engineer?

Pick eight to ten of these for your posting and reword them around your product.

• Design, test and version prompts and structured output schemas for production features.
• Implement tool calling and agent workflows with validated arguments, permission limits and confirmation for side effects.
• Build and maintain evaluation sets, automated scoring and LLM-as-judge rubrics validated against human labels.
• Integrate retrieval (RAG) pipelines with the data and search teams, and measure retrieval quality separately from generation.
• Select models and route requests between hosted and open-weight options on quality, latency and cost.
• Own serving choices, from managed APIs to self-hosted vLLM, including batching, caching and streaming.
• Set and monitor cost and latency budgets per feature; investigate regressions after provider or prompt changes.
• Instrument tracing and observability for every request, including prompt and model versions and tool calls.
• Implement safety controls for prompt injection, output handling and data exposure, and add them to the eval set.
• Run canary releases and rollbacks for prompts and model versions.

Which skills should the job description require?

• Strong Python (or TypeScript) and backend engineering: APIs, async processing, queues, testing.
• Hands-on production experience with at least one major model API and its structured output and tool-calling features.
• Evaluation practice: building golden sets, scoring methods, and knowing the limits of LLM-as-judge.
• RAG fundamentals: chunking, embeddings, hybrid search, reranking and grounding, with a vector store such as pgvector or Qdrant.
• Serving and cost literacy: tokens, context limits, prompt caching, batching, streaming, and when self-hosting pays off.
• Observability: tracing requests with tokens, latency and versions; reading dashboards to find regressions.
• Security awareness: prompt injection, output validation, least-privilege tools, tenant isolation in retrieval.
• Versioning discipline for prompts, schemas and model identifiers, with canary and rollback.
• Clear written communication: design notes, eval reports and decision records.

What is nice to have?

• Fine-tuning open-weight models with LoRA or QLoRA, and the judgement to know when not to.
• Experience running vLLM or another inference server on GPUs, including quantisation.
• Agent frameworks and protocols such as LangGraph, the OpenAI Agents SDK or MCP.
• Domain knowledge in your sector (fintech, healthcare, legal, support operations).

LLM engineer job description template

Copy, replace the bracketed parts and remove anything that is not true for your team.

About the role

We are hiring an LLM Engineer to build and run [product or feature] on [model provider or open-weight stack]. You will own the path from prompt and schema design to evaluation, serving, cost and observability in production, working with [backend, data, product] teams. This is a hands-on role reporting to [manager title].

Responsibilities

• Design, evaluate and version prompts, structured outputs and tool-calling flows.
• Build and maintain evaluation sets and automated scoring; gate releases on them.
• Integrate retrieval and data sources; measure and improve retrieval quality.
• Choose models and serving paths; enforce cost and latency budgets.
• Instrument tracing, monitor quality and investigate regressions.
• Implement and test safety controls, including prompt-injection defences.

Requirements

• [3+] years of backend engineering in Python or TypeScript, with production systems you have operated.
• [1+] years shipping LLM features to real users, with examples you can discuss in detail.
• Experience building evaluation sets and using LLM-as-judge responsibly.
• Working knowledge of RAG pipelines and vector search.
• Understanding of token economics, caching, streaming and serving options.
• Familiarity with the OWASP Top 10 for LLM Applications and practical mitigations.

Nice to have

• LoRA or QLoRA fine-tuning of open-weight models.
• vLLM or similar inference serving on GPUs.
• Agent frameworks and MCP integrations.
• Experience in [your industry].

How we work

[Remote or hybrid], [time zone overlap expected], [engagement: full-time, contract or contract-to-hire], [tools and rituals]. We review prompts and evals the way we review code, and we pin model versions. [Compensation range or rate band], [benefits or contract terms].

How does the role change by seniority?

• Junior: implements prompts, schemas and eval cases to a design someone else owns; strong backend fundamentals matter more than LLM history.
• Mid-level: owns a feature end to end, including its eval set, serving configuration and cost dashboard; can debug a regression without help.
• Senior: makes model, serving and architecture decisions with measured trade-offs, designs evaluation strategy, sets safety controls, and mentors others.
• Lead or staff: sets the platform: shared eval tooling, routing, observability standards, prompt and model release process, and cost governance across teams.

What do LLM engineers earn in 2026?

Quote ranges from named sources and state your own band in the posting. For US full-time roles, KORE1's LLM engineer salary guide (updated July 2026) lists base salaries of $110,000 to $150,000 for associates, $150,000 to $195,000 for mid-level, $195,000 to $265,000 for senior and $250,000 to $340,000 or more for lead and staff. Indeed (updated 12 October 2026) reports an average of $151,505 for AI developers across 3.8k posted salaries, ranging from $92,136 to $249,131. For the adjacent machine learning engineer title, Levels.fyi (updated 17 October 2026) shows median total compensation of $280,000 in the United States, with the 25th percentile at $200,000 and the 75th at $385,000.

For contractors, Lemon.io's Q3 2026 rate report lists remote LLM developers at $31 to $41 per hour for mid-level, $43 to $53 for senior and $52 to $61 for strong senior, while Arc.dev states that AI developers on its platform typically charge $60 to $100 or more per hour. Hyparz publishes indicative hourly bands for AI engineers that track the rates Arc.dev publishes; the quote depends on specialism, seniority and engagement length. The cost guide compares fully loaded employee cost with contractor rates.

What are the common mistakes in LLM engineer job descriptions?

• Asking for model training, research publications and a PhD for a role that builds on hosted APIs; you will lose application engineers and attract researchers who do not want the job.
• Omitting evaluation, cost and observability, which tells strong candidates the team is still prototyping.
• Listing every framework by name instead of the capabilities you need; frameworks change quarterly.
• Requiring "five years of LLM experience" for technology that has been mainstream for three.
• Hiding the model stack, data constraints and whether self-hosting is involved; candidates self-select on this.
• No compensation or rate band, and no statement of remote policy and time zone overlap.

Where should you post, and how do you hire faster?

Post on your careers page, LinkedIn and one or two AI-specific job boards, and ask your current engineers for referrals; the candidates you want are rarely applying cold. Keep the loop to two rounds, as the LLM engineer interview questions guide lays out, and decide before posting whether the work is a scoped build or an ongoing role, since that decides contractor versus full-time.

To skip sourcing and screening, Hyparz matches companies with pre-vetted remote LLM developers typically within days. About 5% of AI-role applicants pass vetting, every engagement starts with a one-week risk-free trial, and you get a dedicated account manager, automated timesheets and source-code protection. Engineers are available as individual experts, blended teams or full remote teams, and the LLM application development service covers scoped builds when you need outcomes rather than a hire.