Senior Product Quality Analyst

Job Summary

Owning the quality bar for Spotify's AI voice models, the full-time Senior Product Quality Analyst will define evaluation criteria, design assessment rubrics, and conduct evaluation campaigns to ensure optimal performance across various products and audiences in a remote capacity.

Key responsibilities

• Define and measure AI voice quality, establishing what constitutes "good enough" for different use cases
• Design evaluation rubrics and conduct qualitative and quantitative assessments to provide actionable insights
• Benchmark AI voices against competitors and collaborate with cross-functional teams to maintain quality standards

Required qualifications

• Experience in product quality, evaluation, or data curation related to model or product performance
• Ability to independently create evaluation criteria and develop repeatable assessment rubrics
• Understanding of voice or audio quality attributes such as naturalness, pacing, and pronunciation
• Strong judgment in balancing product requirements and stakeholder perspectives
• Curiosity about generative AI and speech technology trends, with the ability to translate these into benchmarks