AI Agent Testing Specialist

Job Summary

A company is looking for an Evaluation Scenario Writer - AI Agent Testing Specialist.

Key Responsibilities

• Create structured test cases that simulate complex human workflows
• Define gold-standard behavior and scoring logic to evaluate agent actions
• Analyze agent logs, failure modes, and decision paths

Required Qualifications

• Bachelor's and/or Master's Degree in relevant fields such as Computer Science or Data Science
• Background in QA, software testing, data analysis, or NLP annotation
• Good understanding of test design principles
• Comfortable with structured formats like JSON/YAML for scenario description
• Basic experience with Python and JS