ML Researcher, Foundational Models

About this job

Sarvam AI is creating India's full-stack sovereign AI platform, focusing on research, models, infrastructure, and applications to make artificial intelligence effective for the country. The company collaborates with major enterprises and public institutions. As an ML Researcher for Foundational Models based in Bengaluru, you will tackle open-ended challenges regarding the architecture, optimization, and training dynamics of next-generation models. Unlike fine-tuning positions, this role asks you to explore vague hypotheses, design targeted ablations at scale, and convert your findings into actionable decisions for production training runs. A normal day involves working closely with data and infrastructure teams, running pre-training experiments end-to-end, leveraging direct access to large computing resources, and defending your technical reasoning using empirical evidence.

Responsibilities

• Drive open-ended research on architecture, optimization, scaling, training stability, and post-training recipes.
• Design and execute ablations at scales that inform large-run decisions, including handling pre-training experiments end-to-end.
• Translate research findings into concrete proposals for upcoming training runs and own them through to production.
• Collaborate closely with infrastructure and data teams to address research questions at system boundaries.
• Read broadly, write internal documentation, and publish external findings when appropriate.

Requirements

• PhD in Machine Learning, Computer Science, or a closely related field, or being in the final stages of completion.
• 3+ years of research experience post-PhD or equivalent depth, with consideration for exceptional early-career candidates.
• First-author publications at top-tier ML venues such as NeurIPS, ICML, ICLR, ACL, EMNLP, or COLM.
• Hands-on experience pre-training transformer-based language models from scratch, ideally at 7B+ parameters.
• Meaningful contributions to the open-source LLM ecosystem including research code, model releases, or datasets.
• Fluency in PyTorch with comfort in distributed training and identifying bottlenecks in training loops.
• Strong intuition for experimental design, knowing what to measure, ablate, and trust at scale.

Good to have

• Work on novel architectures like mixture-of-experts, state-space models, or hybrid architectures.
• Experience with multilingual or multimodal pre-training.
• Research contributions in post-training such as RLHF, RLVR, distillation, or reasoning models.
• A track record of taking research ideas from prototypes to shipped capabilities.

Who should apply

This role suits experienced researchers and PhD graduates who possess a strong publication record and hands-on experience in pre-training large language models from scratch. Ideal candidates are comfortable with PyTorch, distributed training, and open-ended experimental design. While a PhD and post-PhD experience are preferred, exceptional early-career candidates with a proven research background may also be considered.

Frequently asked questions