Senior Software Engineer

Job Summary

To evaluate AI coding agents, the freelance Senior Software Engineer will create realistic developer environments, design challenging tasks, and write tests to verify agent solutions while working remotely.

Key responsibilities

• Build realistic developer environments with a codebase, infrastructure, and context
• Design tasks and define evaluation criteria to assess AI agent performance
• Write and iterate on tests based on QA feedback to ensure fair evaluations

Required qualifications

• 5+ years in software development
• Proficiency in Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, and Redis
• Experience writing functional and integration tests
• English proficiency at B2 level or higher
• Ability to complete a technical assessment and identity verification