California Licensed Software Engineer
Get new jobs like this one by email. Create a free account for job alerts and to save the jobs you like.
Job Summary
To support the evaluation and automation of AI tools, the remote California Licensed Software Engineer will build and integrate evaluation harnesses, automate benchmarking processes, and ensure reproducibility of results over a 6-month contract with potential for extension.
Key responsibilities
- Build and integrate evaluation harnesses and automation for software development use cases
- Develop versioned, repeatable processes to evaluate AI tools and models with reproducible environments
- Analyze results from benchmarking runs to improve workflows and ensure reliability
Required qualifications
- Strong software engineering background with experience in automation, developer tooling, or test systems
- Proficient in a general-purpose programming language such as Python, Java, or JavaScript
- Solid working knowledge of Git and containerization with Docker
- Experience with APIs, CI/CD pipelines, and typical engineering workflows
- Understanding of AI evaluation processes and potential pitfalls in measurement validity
Complete Job Description
The complete job description is available to premium members. Premium membership includes:
Full access to 39,036 remote jobs from human-vetted companies, updated daily
Resume Builder - AI-powered tool to craft, enhance, and tailor your resume to a specific job
Twice-monthly live group coaching and the full Remote Career Center
20% member discount on Career Services
Backed by a 30-day money-back guarantee