AI Product Analyst

Job Summary

To support the scaling of AI-powered systems, the full-time AI Product Analyst will analyze production behavior of AI agents and LLM-powered workflows, focusing on identifying failure patterns, measuring quality, and driving improvements in a remote environment.

Key responsibilities

• Analyze production behavior of AI agents and automation systems to identify hidden patterns and actionable insights
• Investigate inconsistencies, hallucinations, and quality degradation across automated user journeys
• Design evaluation approaches and build monitoring systems for measuring AI system quality and reliability

Required qualifications

• 3+ years of experience in analytical roles related to AI systems or complex production systems
• Strong Python and advanced SQL skills for deep analytical investigations
• Understanding of LLM limitations, evaluation challenges, and AI failure modes
• Experience building monitoring systems or analytical frameworks for production systems
• Fluent in Russian and English at B2 level or higher, with the ability to work CET-overlapping hours