Staff Data Scientist
Get new jobs like this one by email. Create a free account for job alerts and to save the jobs you like.
Job Summary
Owning the evaluation strategy for AI agents, the full-time Staff Data Scientist will manage pre-release testing, continuous error analysis, and the improvement loop to enhance agent performance, all while working remotely.
Key responsibilities
- Own the evals strategy, including behavioral testing and production monitoring of agents
- Conduct continuous error analysis to identify failure patterns and rank improvement opportunities
- Transform findings into actionable fixes and new evaluations, measuring results with robust methods
Required qualifications
- BS or MS in a technical field such as Computer Science, Statistics, or Applied Math
- 8+ years of experience delivering data science solutions with measurable business impact
- Hands-on experience with evaluating LLM-based systems, including eval design and judge calibration
- Proficiency in Python and SQL, with familiarity in TypeScript
- Experience with GenAI systems in production, including prompt design and observability tooling
Complete Job Description
The complete job description is available to premium members. Premium membership includes:
Full access to 36,607 remote jobs from human-vetted companies, updated daily
Resume Builder - AI-powered tool to craft, enhance, and tailor your resume to a specific job
Twice-monthly live group coaching and the full Remote Career Center
20% member discount on Career Services
Backed by a 30-day money-back guarantee
Company Company Name
Headquarters Headquarters
Founded Founded
Website
Wikipedia Wikipedia URL
The company description is available to premium members. Go Premium for full access to every job and company profile in our remote jobs database, backed by a 30-day money-back guarantee.
