Inference Optimization Manager

Job Summary

Leading a high-impact team, the full-time Inference Optimization Manager will drive the optimization platform for LLM inference on Modular Cloud, collaborating with product and engineering teams to enhance performance across GPU and ASIC architectures while working remotely from the US or Canada.

Key responsibilities

• Lead the development of the optimization platform to achieve state-of-the-art LLM inference performance
• Own the technical direction of Modular Cloud, ensuring LLM performance remains competitive as the landscape evolves
• Translate customer insights into actionable technical roadmaps, partnering with the GTM team for tailored LLM inference solutions

Required qualifications

• 5+ years of experience in distributed systems or performance engineering, including team leadership
• Proven track record of delivering reusable software tools and libraries across teams
• Strong judgment in evaluating technical tradeoffs and setting priorities
• Ability to translate customer and product needs into focused engineering direction
• Collaborative mindset with a creative approach to complex problem-solving