Technical Product Manager

Job Summary

Owning the accuracy and efficiency of AI for CentrlX, the full-time remote Technical Product Manager will manage evaluation datasets, model selection, and cross-functional partnerships to enhance product quality and drive cost optimization.

Key Responsibilities

• Build and maintain CentrlX's evaluation foundation, including golden datasets and grading rubrics
• Continuously benchmark models across providers and manage model migrations to optimize accuracy and cost
• Serve as the primary contact for AI quality escalations, writing user stories and managing changes for product improvements

Required Qualifications

• 3+ years of experience in product management, data science, or applied AI, with at least 2 years on LLM-based products
• Hands-on experience with Python and SQL for data analysis and metric computation
• Demonstrated ability to build evaluation datasets and harnesses for LLM systems
• Working fluency with at least one eval or LLM observability platform
• Statistical literacy to assess significance and interpret data comparisons accurately