LLMops Engineer

About this job

As an LLMOps Engineer at Coursera in India, you will spend your normal day managing the operational backbone for agentic AI systems, handling tasks like prompt and pipeline versioning, evaluation, monitoring, incident handling, and cost management. You will work within an innovation team that focuses on AI enablement for both internal organizational adoption and customer-facing solutions for enterprise clients. Your daily routine involves building and maintaining infrastructure and tooling that allows non-technical team members to build and deploy tools, ensuring reliable, observable, and cost-effective production systems. Coursera and Udemy operate as a combined platform aiming to help hundreds of millions of learners and thousands of enterprise customers build essential skills for an AI-transformed world.

Responsibilities

• Build and maintain operational tooling for LLM-powered systems including prompt versioning, evaluation harnesses, and CI/CD workflows.
• Implement monitoring and observability to track latency, token usage, cost per request, output quality, and drift.
• Design and run automated evaluation suites to catch regressions, hallucinations, and quality degradation.
• Manage RAG pipelines and vector store infrastructure to keep retrieval sources fresh, accurate, and performant.
• Implement safety and compliance guardrails like content filtering, PII redaction, and access controls.
• Own cost governance for LLM usage through caching strategies, model routing, and usage reporting.
• Collaborate with Product Managers and Senior Engineers to scope operational requirements and build reliable systems.

Requirements

• 4+ years of experience in a software engineering role with a backend engineering and DevOps background.
• Hands-on experience operating or supporting LLM-powered systems in production.
• Proficiency in a backend language like Python, Java, TypeScript, or Go.
• Familiarity with Docker, container orchestration like Kubernetes, and cloud platforms such as AWS, GCP, or Azure.
• Experience implementing APIs, working with SQL and NoSQL databases, and writing automated tests.
• Knowledge of infrastructure-as-code like Terraform and CI/CD tooling like GitHub Actions or Jenkins.

Good to have

• Experience with LLMOps-specific tooling such as LangFuse, LangSmith, or vector databases like Pinecone, Weaviate, and pgvector.
• Working knowledge of RAG architectures, MCP implementation, and agent orchestration frameworks like LangGraph.
• Exposure to prompt management, model access management, and deterministic guardrails treated as code.
• Understanding of AI governance considerations including data privacy, residency, and compliance.

Who should apply

This role suits experienced software engineers with a strong background in backend engineering, DevOps fundamentals, and hands-on experience running production-grade LLM systems. Candidates should be comfortable working in a fast-paced environment, possess a proactive ownership-driven mindset, and have strong communication skills for cross-functional collaboration. Freshers should note that 4+ years of experience is required for this position.

Frequently asked questions