Model Serving Engineer

Job Summary

To support the deployment of high-performance machine learning models, the full-time Model Serving Engineer will design, build, and operate inference platforms remotely, focusing on optimizing performance and ensuring reliability across diverse workloads.

Key Responsibilities

• Design and operate model serving platforms for various machine learning workloads
• Optimize inference performance through techniques such as continuous batching and request multiplexing
• Implement autoscaling and capacity management systems to balance latency, throughput, and cost

Required Qualifications

• Bachelor's or Master's degree in Computer Science or a related field
• Six or more years of experience in distributed systems, infrastructure, or ML platform engineering
• Strong proficiency in Python and a systems language such as Go, Rust, or C++
• Deep experience operating high-throughput, low-latency services in production
• Hands-on experience with large model inference frameworks such as vLLM or TensorRT-LLM