Senior ML Engineer

Job Summary

Driving inference optimization, the full-time Senior ML Engineer will enhance throughput, reduce latency, and improve KV cache utilization in a remote setting, leveraging advanced machine learning techniques and a deep understanding of both hardware and software systems.

Key responsibilities

• Push throughput through continuous batching and kernel-level tuning across various ML frameworks
• Cut latency by profiling and addressing bottlenecks in compute, memory, and networking
• Quantize models without quality regression and optimize KV cache strategies for enhanced performance

Required qualifications

• 5+ years of experience building real ML systems with a focus on inference or training infrastructure
• Strong proficiency in Python for production services
• Hands-on experience with vLLM, SGLang, or TensorRT-LLM
• Fluency in quantization tradeoffs and experience measuring quality regressions
• Comfort with distributed systems, including multi-GPU and multi-node setups