Model Serving Engineer

Job Summary

Model Serving Engineer, responsible for designing and operating high-performance inference platforms for serving large machine learning models in production, working full-time in a remote capacity.

Key Responsibilities

• Design and operate model serving platforms for diverse workloads including LLMs and recommendation systems
• Optimize inference performance through techniques like continuous batching and request multiplexing
• Implement multi-tenant routing and manage GPU utilization for efficient model serving

Required Qualifications

• Bachelor's or Master's degree in Computer Science or a related field
• Six or more years of experience in distributed systems or ML platform engineering
• Proficiency in Python and a systems language such as Go, Rust, or C++
• Experience with high-throughput, low-latency services in production
• Hands-on experience with LLM inference frameworks like vcLLM or TensorRT-LLM