Model Serving Engineer

Job Summary

To support a growing team, the full-time Model Serving Engineer will design, build, and operate high-performance inference platforms for serving large machine learning models remotely.

Key Responsibilities

• Design and operate model serving platforms for diverse workloads including LLMs and vision models
• Optimize inference performance through continuous batching, request multiplexing, and GPU utilization
• Implement and manage multi-tenant routing, autoscaling, and capacity management systems

Required Qualifications

• Bachelor's or Master's degree in Computer Science or a related field
• Six or more years of experience in distributed systems or ML platform engineering
• Strong proficiency in Python and a systems language such as Go, Rust, or C++
• Deep experience operating high-throughput, low-latency services in production
• Hands-on experience with LLM or large model inference frameworks