Model Serving Engineer

Job Summary

Model Serving Engineer, a full-time remote position focused on designing, building, and operating high-performance inference platforms for machine learning models, requiring extensive experience in distributed systems and performance engineering.

Key Responsibilities

• Design and operate model serving platforms for diverse workloads
• Optimize inference performance and implement multi-tenant routing and quality-of-service policies
• Drive end-to-end observability and develop deployment workflows for high-availability AI services

Required Qualifications

• Bachelor's or Master's degree in Computer Science or a related field
• Six or more years of experience in distributed systems or ML platform engineering
• Strong proficiency in Python and a systems language such as Go, Rust, or C++
• Hands-on experience with large model inference frameworks
• Deep understanding of GPU architecture and performance engineering