Software Engineer

Job Summary

To support the expansion of the AI Benchmarking Hub, the full-time remote Software Engineer will implement and develop AI benchmarks, maintain benchmarking infrastructure, and collaborate with cross-functional teams to ensure accurate evaluation data and insights.

Key Responsibilities

• Implement AI benchmarks within the evaluation infrastructure to expand capabilities and evaluate new model releases
• Contribute to the development of new benchmarks, pitching and prototyping innovative ideas
• Collaborate with researchers, analysts, and engineers to ensure the accuracy and integration of evaluation data into research products

Required Qualifications

• More than two years of professional experience in software engineering with a strong background in building and maintaining complex systems
• Ability to generate ideas for new benchmarks and experiments
• Motivation aligned with the mission to provide independent insights into AI trends
• Familiarity with evaluation frameworks like Inspect and current AI trends is a plus