Staff Backend Engineer - K8 (Envoy)

About this job

Coupang is a fast-growing e-commerce company on a mission to reshape the future of commerce by making shopping, eating, and living easier. Operating with a unique blend of startup culture and the resources of a global public company, the firm empowers engineers to build bold and impactful solutions. As a Staff Backend Engineer for K8 (Envoy) based in Bengaluru, you will work alongside platform and product leaders to design and deliver robust infrastructure solutions. Your main objective is to drive the development of scalable, reliable platform services and shape Coupang's next-generation AI Inference Gateway platform. A normal day in this role involves designing high-performance request routing, establishing multi-tenant platform capabilities, collaborating closely with machine learning and data teams, and leading architecture reviews to ensure optimal system performance and operational excellence for mission-critical AI workloads.

Responsibilities

• Architect and build Coupang's next-generation AI Inference Gateway platform for machine learning and generative AI workloads.
• Design request routing, model endpoint abstraction, traffic shaping, load balancing, failover, caching, and policy enforcement.
• Drive the technical vision for scalable AI inference infrastructure across cloud and on-prem environments.
• Develop critical infrastructure components in Go, Java, or Python focusing on performance and resiliency.
• Design multi-tenant platform capabilities including authentication, authorization, quota management, and rate limiting.
• Partner with ML Platform, Model Serving, Data, and Product Engineering teams to enable seamless AI workload operations.
• Lead architecture reviews, raise engineering standards, and mentor senior engineers across teams.
• Optimize system performance, latency, throughput, and infrastructure efficiency for large-scale inference workloads.

Requirements

• 8+ years of professional software development experience.
• 5+ years of experience designing and operating large-scale distributed systems in production.
• Strong hands-on programming expertise in Go, Java, or Python.
• Proven track record of building highly available, mission-critical platform or infrastructure services.
• Experience designing API platforms, service gateways, service mesh, or large-scale networking infrastructure.
• Deep understanding of microservices architecture, distributed systems design, and cloud-native technologies.
• Experience with Kubernetes and containerized workloads in production environments.
• Experience operating services on AWS, Azure, or GCP.

Good to have

• Experience building AI/ML inference platforms, LLM gateways, or model serving infrastructure.
• Deep expertise in Kubernetes ecosystem technologies such as Gateway API, Ingress Controllers, and Service Mesh like Envoy.
• Experience with inference serving frameworks like vLLM, Triton Inference Server, or KServe.
• Understanding of high-performance networking, gRPC, HTTP/2, and streaming protocols.
• Experience with distributed data systems such as Kafka, Cassandra, Redis, or MongoDB.

Who should apply

This role suits experienced backend and infrastructure engineers with a strong background in distributed systems, cloud-native technologies, and Kubernetes. Ideal candidates will possess extensive programming expertise in Go, Java, or Python, and have a proven history of building highly available platform services. Because this is a senior staff-level position requiring over eight years of professional experience, freshers are not eligible to apply.

Frequently asked questions