Senior Software Engineer

Job Summary

Seeking a full-time Senior Software Engineer to join the AI Inference Data Plane team, where the successful candidate will design, develop, and deliver high-scale data plane services that support AI-driven applications in a remote environment.

Key responsibilities

• Drive the end-to-end design, development, and delivery of critical data plane components for large generative AI models
• Architect and refine system designs for a multi-tenant AI inference cloud ecosystem, ensuring high availability and resiliency
• Implement and optimize distributed inference hosting, focusing on performance enhancement techniques and collaboration with cross-functional teams

Required qualifications

• Hands-on experience with large language or multimodal models using inference engines like vLLM or TensorRT
• Familiarity with distributed inference serving frameworks such as llm-d or Ray Serve
• Expert-level proficiency in GoLang or Python, including experience with gRPC
• Proven track record of contributions to open-source projects related to AI inference
• Experience in shipping customer-facing software products in a high-scale cloud environment