AI Agent Developer Job Description Template (2026)

Full JD (copy-ready)

Paste this into your ATS or careers page. Edit the company name and any bracketed placeholders.

# AI Agent Developer — Job Description

## Role summary
An AI Agent Developer builds production LLM-powered agents: multi-step tool-calling flows in LangGraph, LangChain, CrewAI, or the Vercel AI SDK, RAG systems backed by Pinecone/Weaviate/pgvector, structured outputs with Pydantic or Zod, evals with golden datasets and LLM-as-judge, and cost/latency monitoring. This is a software engineering role in Python or TypeScript — the bar is a reliably running service, not a notebook demo or a prompt.

## Responsibilities
- Design agent architectures — single-agent vs multi-agent, planner/executor, graph-based state machines — based on the task and latency budget.
- Build RAG pipelines with chunking, embedding (text-embedding-3-large, Voyage, Cohere), hybrid retrieval (BM25 + vector), and re-ranking (Cohere Rerank, Jina) tuned to the content domain.
- Integrate Claude Sonnet 4.5, GPT-4o, Gemini 1.5 Pro, and open-source models (Llama 3.1, Mistral) via OpenAI-compatible APIs with streaming and function/tool calling.
- Route requests across models by task complexity, latency, and cost — cheap models for extraction, frontier for reasoning.
- Enforce structured output with Pydantic or Zod schemas, OpenAI structured outputs, or Instructor; handle validation failures with retries.
- Build eval harnesses with golden datasets (50-500 representative inputs), LLM-as-judge scoring, exact-match checks, and regression tracking on every prompt change.
- Ship guardrails: input classification, output policy checks, PII redaction, grounding verification against retrieved context, and escalation paths to humans.
- Optimize cost with prompt caching (Anthropic prompt caching, OpenAI predicted outputs), embedding caching, context truncation, and small-model routing.
- Deploy agents on Vercel, Modal, Railway, AWS Lambda, or Kubernetes with proper timeouts, retries, and circuit breakers across LLM providers.
- Stream responses to the client with SSE or WebSockets for chat UIs; handle partial outputs, interruptions, and cancellations cleanly.
- Instrument every call with workflow tags, user IDs, token counts, and latency; log to LangSmith, Langfuse, Helicone, or a custom Postgres table.
- Run sandboxed tool execution (code interpreter, browser agents via Playwright/Browserbase, shell) with resource limits and output validation.

## Must-have skills
- 3+ years software engineering in Python or TypeScript with production services.
- Shipped at least one LLM-powered agent to real users — not a tutorial, not a notebook.
- Hands-on with LangChain/LangGraph, LlamaIndex, CrewAI, AutoGen, or the Vercel AI SDK in production.
- Deep familiarity with at least two LLM APIs (OpenAI, Anthropic, Google) including streaming, tool calling, and structured outputs.
- RAG experience across embedding choice, chunking strategy, hybrid search, and re-ranking.
- Vector DB production experience — Pinecone, Weaviate, pgvector, Qdrant, or Chroma.
- Eval discipline: has built a golden-set + LLM-judge pipeline and used it to catch regressions.
- FastAPI, Next.js API routes, or similar for shipping the agent as a service.

## Nice-to-have skills
- Fine-tuning (LoRA/QLoRA) open-source models on Modal, Together, or Hugging Face.
- Voice agents on Vapi, Retell, or LiveKit with Deepgram STT and ElevenLabs TTS.
- Browser automation agents with Playwright, Browserbase, or Anthropic Computer Use.
- Knowledge graph retrieval (Neo4j, GraphRAG) for relational document sets.
- DSPy or TextGrad for prompt optimization over eval datasets.
- Observability tools (LangSmith, Langfuse, Helicone, Phoenix/Arize) in production.

## Tools and technology
- Python / TypeScript
- LangChain / LangGraph
- Vercel AI SDK
- OpenAI / Anthropic / Google SDKs
- Pinecone / Weaviate / pgvector
- Pydantic / Zod
- FastAPI / Next.js
- LangSmith / Langfuse
- Modal / Railway
- Docker

## Reporting structure
Reports to the Head of AI, Engineering Manager, or CTO depending on org size. Partners daily with product (flow design and user research), software engineers (integration surface, auth), design (chat UI and failure states), and whoever owns the content/knowledge corpus that feeds the RAG layer.

## Success metrics (KPIs)
- Task success rate on the golden eval set — trending up release over release with no regressions shipping to prod.
- Hallucination rate on grounded answers measured against retrieved context — held under a documented threshold.
- P95 end-to-end latency within SLA for the agent flow (streaming first-token and full-response targets).
- Cost per user session (LLM + retrieval + infra) trending flat or down quarter-over-quarter.
- Production incident rate (rate limits, timeout, schema validation failures) trending down.
- Human-escalation rate on agent flows with HITL — within target range, neither rubber-stamping nor drowning humans.