Software Engineering Specialist

Job Summary

To support innovative AI projects, the part-time Software Engineering Evaluation Specialist will design coding tasks within self-contained Docker environments, focusing on creating realistic developer scenarios that challenge AI coding agents.

Key responsibilities

• Invent realistic developer scenarios and build reproducible Docker environments with pinned dependencies
• Write pytest tests to verify outcomes and create instructional documentation resembling developer tickets
• Calibrate task difficulty for AI agents and iterate based on feedback from expert QA reviewers

Required qualifications

• 3+ years of production software development experience in a backend stack such as Python, Go, Node.js, Java, or Rust
• Fluency in Python and pytest, with experience in creating tests and debugging
• Proficiency in Docker authoring and creating reproducible Dockerfiles
• Comfort with Linux and Bash for debugging within containers
• Experience working with AI coding agents on non-trivial tasks