Senior Product Manager, AI Evaluation

Job Summary

Driving the evaluation strategy for AI specialists, the full-time Senior Product Manager, AI Evaluation will define methodologies, develop a closed-loop evaluation platform, and influence cross-functional teams in a flexible work environment.

Key responsibilities

• Define the end-to-end evaluation methodology for AI specialists, including metrics and failure taxonomy
• Develop a continuous improvement engine for evaluation that utilizes production telemetry for quality enhancement
• Collaborate with a federated Center of Excellence to establish reusable evaluation infrastructure and tooling across business units

Required qualifications

• 12+ years of software product management experience
• Deep experience in AI/ML product management, particularly with LLM evaluation and applied ML quality frameworks
• Proven track record of defining methodologies in ambiguous, cross-team environments
• Strong technical fluency in evaluation pipelines and production telemetry
• Ability to drive outcomes through influence and technical credibility without formal authority