Senior Engineer, Inference Data Plane

Job Summary

Seeking a full-time Senior Engineer, Inference Data Plane, who will be a key technical leader responsible for designing and delivering high-scale data plane services for AI-driven applications, working remotely.

Key responsibilities

• Drive the end-to-end design, development, and delivery of critical data plane components for large generative AI models
• Architect system design proposals for a high-scale, multi-tenant AI inference cloud ecosystem
• Implement and optimize distributed inference hosting using advanced techniques like tensor/data parallelism and smart routing

Required qualifications

• Hands-on experience with large language or multimodal models using inference engines like vLLM or TensorRT
• Familiarity with distributed inference serving frameworks such as llm-d or NVIDIA Dynamo
• Expert-level proficiency in GoLang or Python, with familiarity in gRPC
• Proven experience in shipping customer-facing software products in a high-scale environment
• Experience contributing to open-source projects related to inference engines or frameworks