AI Evaluation Specialist
Get new jobs like this one by email. Create a free account for job alerts and to save the jobs you like.
Job Summary
Evaluating enterprise AI tools and capabilities, the full-time AI Evaluation Specialist will assess effectiveness, safety, usability, and readiness for business adoption while working remotely.
Key responsibilities
- Plan and execute hands-on evaluations of AI tools using representative business scenarios and workflows
- Design and maintain standardized evaluation frameworks, scorecards, and acceptance criteria for AI tools and models
- Conduct comparative evaluations and proof-of-concept assessments across multiple AI products and vendors
Required qualifications
- 5+ years of experience in model evaluation and quality assurance of AI-enabled products or services
- Experience developing structured evaluation plans, scoring rubrics, and measurable acceptance criteria
- Hands-on experience with generative AI tools and large language models, including prompt design and output analysis
- Familiarity with common generative AI architectures and capabilities, including APIs and enterprise search
- Bachelor's degree
Complete Job Description
The complete job description is available to premium members. Premium membership includes:
Full access to 39,386 remote jobs from human-vetted companies, updated daily
Resume Builder - AI-powered tool to craft, enhance, and tailor your resume to a specific job
Twice-monthly live group coaching and the full Remote Career Center
20% member discount on Career Services
Backed by a 30-day money-back guarantee
Company Company Name
Headquarters Headquarters
Founded Founded
Website
The company description is available to premium members. Go Premium for full access to every job and company profile in our remote jobs database, backed by a 30-day money-back guarantee.
