Senior Python Engineer

Job Summary

Experienced Senior Python Engineers will find a freelance opportunity focused on evaluating AI coding agents by creating realistic developer environments, designing tasks, and writing tests to assess agent solutions, all while working remotely.

Key responsibilities

• Build realistic developer environments that simulate a virtual company with a comprehensive codebase and context
• Design tasks and evaluation criteria to assess how well AI agents handle real-world developer tasks
• Iterate on tasks and tests based on QA feedback to ensure fair and robust evaluations

Required qualifications

• 5+ years of professional experience with Python
• Strong experience with FastAPI, pytest, and async/await
• Hands-on experience with Docker, PostgreSQL, and CI/CD pipelines
• Proven experience writing and maintaining automated tests
• English proficiency at B2 level or higher