Machine Learning Systems Engineer

Job Summary

To support AI Safety and Security Engineering initiatives, the full-time Evaluation and ML Systems Engineer will design evaluation infrastructure, define metrics and protocols, and ensure traceability of results, working remotely or onsite in select locations.

Key responsibilities:

• Build the benchmarking and reproducibility systems necessary for evaluation
• Define metrics and protocols for measuring performance
• Map results to the code and runs that produced them to ensure traceability

Required qualifications:

• Bachelor's degree (or equivalent experience) with 5+ years in ML engineering or evaluation
• Experience designing benchmarks, metrics, and statistically sound comparisons for ML systems
• Demonstrated measurement rigor with a careful approach to metrics and claims
• Strong Python engineering skills for developing shared infrastructure