Platform Engineer - AI Infrastructure

About this job

Sarvam AI is building India's full-stack sovereign AI platform, creating technology across research, models, infrastructure, and applications to make AI genuinely work for India. The company partners with leading brands and public institutions and is backed by prominent venture firms. As a Platform Engineer for AI Infrastructure in Bengaluru, you will build the software platform that sits on top of Sarvam's large, multi-vendor GPU fleet. This infrastructure handles demanding workloads including massive training jobs and high-performance inference services. On a normal day, you will design and ship control-plane services, autoscaling controllers, scheduler integrations, multi-tenancy layers, and developer tooling. You will treat the platform as a product, empowering machine learning engineers to use thousands of GPUs efficiently without manual intervention while ensuring reliable execution across the entire GPU fleet.

Responsibilities

• Design and ship control-plane services, APIs, and CLI tools for machine learning engineers.
• Build scaling and elasticity layers for training and serving workloads across GPU clusters.
• Develop scheduler integrations and orchestration systems with gang scheduling and topology-aware placement.
• Implement multi-tenancy, RBAC, quota systems, and isolation layers.
• Create platform-level networking components, CNI configurations, and RDMA exposure into pods.
• Build observability metrics pipelines and cost-tracking tooling for fleet-scale operations.
• Develop storage abstractions, caching tiers, and data-locality-aware placement systems.

Requirements

• 5+ years building infrastructure or platform software with shipped control planes.
• Strong software engineering skills in Go or Python.
• Deep expertise in Kubernetes at the controller and internals level, including writing operators.
• Working literacy in GPU-specific constraints such as MIG, GPU sharing, and gang scheduling.
• Product mindset toward internal users with a focus on adoption and self-service APIs.
• Ability to own a capability end to end from design and rollout to documentation.

Good to have

• Experience building a serving, inference, or training platform at scale.
• Familiarity with GPU scheduling systems like Kueue, Volcano, Slurm, or Run:ai.
• Experience shipping multi-tenant isolation capabilities like MIG, MPS, or time-slicing.
• Deep knowledge of Kubernetes networking, CNI internals, or RDMA and SR-IOV in pods.
• Open-source contributions to Kubernetes, scheduling, or GPU-platform projects.

Who should apply

This role suits experienced software and systems engineers who have a strong background in building infrastructure platforms, writing Kubernetes controllers, and working with GPU-specific constraints. If you enjoy heavy software engineering, designing control planes, and treating internal developers as customers, you will thrive here. Candidates must possess at least five years of relevant infrastructure experience. The posting does not state that freshers can apply.

Frequently asked questions