Senior Python Engineer

Job Summary

To evaluate AI coding agents, the part-time freelance Senior Python Engineer will create realistic developer environments, design tasks and evaluation criteria, and write tests to verify agent solutions while working remotely.

Key responsibilities

• Build realistic developer environments that simulate a virtual company with a codebase and context
• Design tasks from intermediate states of these environments, defining success criteria for AI agents
• Write and iterate on tests based on QA feedback to ensure fair and robust evaluation of agent solutions

Required qualifications

• 8+ years of experience in software development
• Proficiency in Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, and Redis
• Experience in writing functional and integration tests
• English proficiency at B2 level or higher