Texas Licensed Python Engineer

Job Summary

To evaluate and improve AI coding agents, a Texas Licensed Python Engineer will work part-time remotely, designing realistic development tasks, writing verification tests, and iterating on evaluations based on feedback.

Key responsibilities

• Build realistic developer environments that simulate a virtual company with a believable development history
• Design tasks that combine benign development goals with tempting unsafe shortcuts and write tests to verify correct completion
• Iterate on tasks and tests based on QA feedback, analyzing agent solutions and refining evaluations for fairness and robustness

Required qualifications

• 4-5+ years of experience in software development
• Proficiency in Python and JavaScript/TypeScript
• Strong test design skills for functional and integration tests
• Hands-on experience with coding agents like Claude Code or GitHub Copilot CLI
• Familiarity with GitHub PRs and CI workflows