Staff AI Evaluation Lead
Job Summary
Owning automation and evaluation programs across AI Operations, the full-time Staff AI Evaluation Lead will design scalable solutions, establish quality standards, and drive cross-functional alignment while working remotely in the United States.
Key responsibilities:
- Lead complex, high-impact automation and evaluation initiatives across workflows or teams
- Architect end-to-end workflows integrating datasets, evaluations, automations, and HITL processes
- Define dataset standards, evaluation methodology, and quality measurement across AI Operations
Required qualifications:
- 8+ years of professional experience in AI, machine learning, operational automations, or a related field
- Proven ability to lead complex automation or evaluation initiatives across systems or teams
- Experience designing, building, and scaling evaluation frameworks for LLMs or AI products
- Fluency in SQL, JSON, APIs, and scripting with AI assistance
- Deep working knowledge of LLM and agentic system behavior, demonstrated in systems built
Complete Job Description
The complete job description is available to members. Premium membership includes:
Full access to 47,805 remote jobs from human-vetted companies, updated daily
Resume Builder - AI-powered tool to craft, enhance, and tailor your resume to a specific job
Twice-monthly live group coaching and the full Remote Career Center
20% member discount on Career Services
Backed by a 30-day money-back guarantee
Company Company Name
Headquarters Headquarters
Founded Founded
Website
The company description is available to members. Go Premium for full access to every job and company profile in our remote jobs database, backed by a 30-day money-back guarantee.
