Senior Automation Specialist

Job Summary

A company is looking for a Senior Automation Specialist, AI Operations.

Key Responsibilities

• Own evaluation and datasets by defining approaches and ensuring coverage of real-world scenarios
• Build evaluation systems to measure model quality and performance through manual and automated evaluations
• Translate ambiguous problems into structured evaluation frameworks and execution plans

Required Qualifications

• 3+ years of experience with evaluation frameworks, datasets, or quality measurement systems
• Experience in building or managing datasets, including labeling and QA
• Ability to independently drive tasks from problem definition to execution
• Hands-on experience with AI tools, LLM workflows, or automation platforms
• Basic scripting or data skills (SQL, Python, etc.)