Inference Optimization Engineer

Job Summary

To support the optimization of inference performance across a unified cloud stack, the full-time Inference Optimization Engineer will analyze customer workloads, develop an optimization platform, and collaborate with engineering and product teams while working remotely from the US or Canada.

Key responsibilities

• Build the optimization platform to enhance inference performance of LLMs on Modular Cloud across various architectures
• Shape the technical direction of Modular Cloud to maintain LLM performance on the Pareto frontier for evolving use cases
• Collaborate with the GTM team to customize LLM inference for specific customer needs and translate insights into technical direction for engineering

Required qualifications

• 5+ years of experience in distributed systems or performance engineering
• Proven track record of developing reusable software tools and libraries
• Strong judgment in evaluating technical tradeoffs and setting priorities
• Creative problem-solving skills with a collaborative mindset
• Alignment with company culture and values