Model Serving Engineer

Job Summary

Model Serving Engineer, a full-time remote position requiring over six years of experience, is responsible for designing and operating high-performance inference platforms for machine learning models in production.

Key Responsibilities

• Design and operate model serving platforms for diverse workloads, including LLMs and recommendation systems
• Optimize inference performance through various techniques, including batching and request multiplexing
• Implement security controls and ensure end-to-end observability of AI services

Required Qualifications

• Bachelor's or Master's degree in Computer Science or a related field
• Six or more years of experience in distributed systems or ML platform engineering
• Strong proficiency in Python and a systems language such as Go, Rust, or C++
• Deep experience with high-throughput, low-latency services in production
• Hands-on experience with large model inference frameworks like vLLM or TensorRT-LLM