Senior Python Engineer

Job Summary

To support the evaluation of AI coding agents, the part-time freelance Senior Python Engineer will create realistic developer environments, design challenging tasks, and write tests to verify AI solutions while working remotely.

Key responsibilities

• Build realistic developer environments with a virtual company setup, including codebase and infrastructure
• Design tasks and evaluation criteria that challenge AI agents, ensuring tasks are solvable and well-defined
• Write and iterate on tests that verify agent solutions, refining them based on QA feedback to ensure fairness and robustness

Required qualifications

• 8+ years of experience in software development
• Proficiency in Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, and Redis
• Experience writing functional and integration tests
• English proficiency at a B2 level or higher