Principal Machine Learning Engineer

Job Summary

Leading the technical direction for AI agent quality in Studio, the full-time Principal Machine Learning Engineer will define measurement standards, design evaluation systems, and manage the quality architecture for agent performance in a remote work environment.

Key responsibilities

• Own the agent quality architecture by defining measurement criteria and production traffic sampling methods
• Design evaluation systems focused on cost and accuracy, incorporating deterministic checks and behavioral heuristics
• Lead hands-on solutions for complex quality challenges, ensuring effective agent performance and reliability

Required qualifications

• 6+ years of experience in ML/AI engineering with expertise in LLMs, NLP, or recommendation systems
• Proven track record of building and shipping LLM-powered systems with measurable improvements
• Strong evaluation and experimental skills, including designing benchmarks and assessing quality
• Robust software and data engineering skills, particularly in data pipelines and model serving
• Ability to influence technical direction across teams and mentor senior engineers