Senior Python Engineer

Job Summary

To evaluate AI coding agents, the freelance Senior Python Engineer will create realistic developer environments, design evaluation tasks, and write tests to verify agent solutions while working remotely with a commitment of 30+ hours per week.

Key responsibilities

• Build realistic developer environments that simulate a virtual company with context and a codebase
• Design tasks and criteria for evaluating AI solutions, ensuring they are solvable by the agents
• Write and iterate on tests based on QA feedback to ensure fair and robust evaluation of agent solutions

Required qualifications

• 5+ years of professional experience with Python
• Strong experience with FastAPI, pytest, and async/await
• Hands-on experience with Docker, PostgreSQL, and CI/CD pipelines
• Proven experience writing and maintaining automated tests
• English proficiency at B2 level or higher