AI Product Analyst

Job Summary

To support the evaluation and improvement of AI agents and LLM-powered workflows, the full-time AI Product Analyst will analyze production behavior, investigate inconsistencies, and design monitoring systems while working remotely.

Key responsibilities

• Analyze production behavior of AI agents and automation systems to identify failure patterns and quality issues
• Design evaluation approaches and build monitoring systems for measuring AI system quality and reliability
• Collaborate with cross-functional teams to support decision-making and assess the impact of AI systems on business metrics

Required qualifications

• 3+ years of experience in analytical roles related to AI systems or complex production systems
• Strong Python and advanced SQL skills for data analysis and investigation
• Understanding of LLM limitations and AI failure modes
• Experience building monitoring systems or analytical frameworks for production systems
• Fluent in Russian and English (B2 level or higher), with the ability to work CET-overlapping hours