Senior Machine Learning Engineer

Job Summary

To support AI Safety and Responsibility efforts, the full-time Senior Machine Learning Engineer will design safety benchmarks, develop evaluation methodologies, and build reproducible evaluation pipelines for large language and multimodal models, working remotely or onsite in Santa Clara.

Key responsibilities

• Design and maintain safety benchmarks for large language and multimodal models, translating risk scenarios into test cases and scoring rubrics
• Define metrics and protocols to assess model safety and usefulness, using baselines and failure analysis to identify meaningful improvements
• Develop reusable Python pipelines for reproducible evaluations, including versioned datasets and experiment tracking

Required qualifications

• Bachelor's degree or equivalent experience with 6+ years in machine learning engineering or evaluation
• Experience in developing benchmarks and metrics for ML systems, with the ability to create testable experiments
• Strong Python programming skills and experience in building data pipelines and experiment-tracking workflows
• Ability to assess dataset quality and support conclusions with reproducible evidence
• Clear communication skills to explain findings and tradeoffs to researchers and decision-makers