Remote Jobs Sign In

AI Agent Evaluation Engineer

Get new jobs like this one by email. Create a free account for job alerts and to save the jobs you like.

Location: Remote
Compensation: To Be Discussed
Reviewed: Thu, Nov 12, 2026
This job expires in: 30 days

Job Summary

Working remotely, the full-time AI Agent Evaluation Engineer will focus on evaluating and enhancing AI agents, ensuring optimal performance and user experience across various customer interactions.

Key responsibilities
  • Conduct assessments of AI agent performance and provide actionable feedback for improvements
  • Collaborate with cross-functional teams to develop evaluation criteria and methodologies
  • Analyze conversation data to identify trends and insights that inform AI training and development
Required qualifications
  • Experience in AI or machine learning evaluation methodologies
  • Strong background in data analysis and statistical techniques
  • Familiarity with conversational AI technologies and frameworks
  • Degree in Computer Science, Engineering, or a related field
  • Proficiency in programming languages such as Python or Java

Complete Job Description

The complete job description is available to premium members. Premium membership includes:

Full access to 39,452 remote jobs from human-vetted companies, updated daily

Resume Builder - AI-powered tool to craft, enhance, and tailor your resume to a specific job

Twice-monthly live group coaching and the full Remote Career Center

20% member discount on Career Services

Backed by a 30-day money-back guarantee