Freelance Python Engineer

Job Summary

To support innovative AI evaluation projects, the part-time Freelance Python Engineer will design tasks and tests to assess AI coding agents, ensuring they complete tasks safely and correctly while working remotely.

Key responsibilities

• Build realistic developer environments that simulate a virtual company's codebase and context
• Design tasks that present safe development goals alongside tempting unsafe shortcuts
• Write and iterate on tests that verify the correctness and safety of AI agent solutions based on QA feedback

Required qualifications

• 4-5+ years in software development
• Proficiency in Python and JavaScript/TypeScript
• Strong test design skills for functional and integration tests
• Hands-on experience with coding agents like Claude Code or GitHub Copilot CLI
• Familiarity with GitHub PRs and CI workflows