ML Ops Engineer, Chanakya

About this job

Sarvam AI is creating India's full-stack sovereign AI platform, focusing on research, models, infrastructure, and applications to make artificial intelligence truly work for the country. As an ML Ops Engineer for the Chanakya role, you will take charge of the entire model lifecycle across defense and strategic sector deployments. Your work involves managing serving infrastructure, building monitoring systems, and handling evaluation pipelines and environment management. A normal day requires you to ensure that deployment systems remain consistently active, accurate, and auditable while supporting field engineers and product engineering teams. Because model failures here represent operational risks rather than simple user experience issues, you will operate under high standards to maintain system reliability across both cloud and restricted environments.

Responsibilities

• Design and operate model serving infrastructure across cloud and on-premise environments.
• Build and maintain CI/CD pipelines for evaluation-gated deployments, model updates, and rollbacks.
• Monitor production model performance metrics like latency, throughput, accuracy drift, and failure modes.
• Develop evaluation infrastructure, including A/B testing, model comparison tools, and harnesses.
• Manage containerised model serving for air-gapped, edge, and constrained settings.
• Collaborate with Data Scientists on evaluation pipelines while owning the underlying infrastructure.
• Create operational playbooks and runbooks for Strategic Deployment Engineers to use in the field.
• Handle incident response for failures at the model layer across active deployments.

Requirements

• 3 to 5 years of experience in MLOps or ML engineering with at least one production ML or LLM system running continuously.
• Deep expertise in model serving tools like TGI, vLLM, Triton Inference Server, or equivalent, plus quantised formats such as GPTQ, AWQ, and GGUF.
• Experience adapting and fine-tuning models in constrained, air-prem, or air-gapped environments along with compute and data pipeline management.
• Containerisation experience utilizing Docker, Kubernetes, or lightweight alternatives like K3s and K0s for edge and constrained environments.
• Familiarity with monitoring and observability tools such as Grafana and Prometheus to build custom evaluation dashboards.
• Python fluency along with familiarity with model evaluation frameworks and fine-tuning workflows.
• Hands-on experience utilizing CI/CD tooling for machine learning pipelines like GitHub Actions, DVC, or ArgoCD.

Who should apply

This role suits professionals with several years of focused experience in machine learning operations and production model maintenance. Candidates who have successfully kept production ML systems running under heavy load and built proactive monitoring systems will find a great fit here. You should value high ownership, operational reliability, and documentation. The posting does not mention if freshers can apply, as it specifically requires 3 to 5 years of relevant engineering experience.

Frequently asked questions