ML Engineer (Training Infra), Foundational Models

About this job

Sarvam AI is creating India's full-stack sovereign AI platform, focusing on research, models, infrastructure, and applications to make artificial intelligence work effectively for the country. Backed by major investors and partnering with leading brands, the company aims to build the bedrock of Sovereign AI. As an ML Engineer for Training Infra on the Foundational Models team, you will own the infrastructure used to train the next family of foundational models. This is a deep systems position centred on distributed training at scale, parallelism strategies, GPU kernel work, and stability optimization. A normal day involves building and maintaining the distributed training stack across large GPU clusters, profiling end-to-end training throughput, ensuring the reliability of long-running training jobs, and collaborating closely with researchers and data teams to prevent performance bottlenecks.

Responsibilities

• Build and maintain the distributed training stack across large GPU clusters.
• Design and implement various parallelism strategies for different model architectures and scales.
• Profile and optimize end-to-end training throughput including kernels, memory layout, and data loading.
• Write and tune custom GPU kernels using CUDA or Triton to maximize performance.
• Manage the reliability of long-running training jobs through fault tolerance and automated detection.
• Partner with researchers and the data team to ensure efficient model training without bottlenecks.

Requirements

• BS or MS in Computer Science or a closely related technical field, or equivalent demonstrated experience.
• 3+ years of experience building ML training infrastructure or large-scale distributed systems, with openness to exceptional early-career candidates.
• Hands-on experience training large models using distributed training frameworks like Megatron-LM, DeepSpeed, FSDP, or NeMo.
• Deep working knowledge of GPU architecture, the CUDA programming model, and workload profiling tools.
• Strong knowledge of PyTorch internals and comfort reading and modifying low-level training code.
• Meaningful open-source contributions in the training infrastructure ecosystem.

Good to have

• Custom CUDA or Triton kernel development with measurable performance wins.
• Direct experience training models at 10B+ parameters or on 1000+ GPU clusters.
• Familiarity with cluster orchestration and job scheduling at scale using Slurm or Kubernetes.
• Experience with mixed precision, quantization-aware training, or other production numerical work.
• First-author papers or technical reports on training systems, scaling, or model efficiency.

Who should apply

This role suits systems-focused engineers with a strong background in machine learning training infrastructure, distributed systems, and GPU architecture. Candidates should have practical experience managing large-scale pretraining runs and tackling complex throughput or reliability challenges. Exceptional early-career candidates with a solid systems background are also considered, as well as those with relevant open-source contributions in the training ecosystem.

Frequently asked questions