Software Engineer Task Author

Job Summary

Working remotely on a flexible contract basis, the Software Engineer Task Author will design and calibrate authentic engineering challenges, focusing on diagnosing integration failures and writing verifiable fixes to evaluate advanced AI agents.

Key Responsibilities

• Author realistic engineering task prompts covering integration failures and debugging scenarios
• Write scoring rubrics to define correct fixes and validate them in live environments
• Calibrate task difficulty by adjusting code complexity and failure modes to challenge AI models effectively

Required Qualifications

• Working software engineer with hands-on, industry-level experience
• Fluency in Git, version control workflows, and code review practices
• Strong debugging skills, including reading logs and tracing failures through codebases
• Experience with infrastructure, integrations, and configuration code (APIs, services, CI/CD)
• Ability to write and verify code fixes in live or simulated environments