AI Research Engineer

Job Summary

To enhance the evaluation and performance of AI agents across the business, the full-time AI Research Engineer will build evaluation frameworks, design experiments, and collaborate with cross-functional teams, all while working remotely.

Key responsibilities

• Build evaluation frameworks for AI agents, including datasets and metrics to measure performance on real-world tasks
• Design experiments to compare models and tools, identifying performance trade-offs and understanding agent failures
• Collaborate with teams to translate qualitative feedback into measurable research questions and outcomes

Required qualifications

• Hands-on experience with LLM models or agents through professional work, applied research, or substantial personal projects
• Strong Python skills and solid software engineering fundamentals, with an emphasis on production-quality code
• Experience in applied research, machine learning, or data science, including designing experiments and analyzing datasets
• Good understanding of LLM agent functionalities beyond prompting, including tool use and common failure modes
• Ability to independently tackle open-ended problems and develop testable questions