Staff Applied Scientist

Job Summary

A company is looking for a Staff Applied Scientist - AI Evaluation & Trust.

Key Responsibilities

• Lead the development of specialized judge models and evaluation frameworks
• Design and execute rigorous scoring pipelines and calibrations for agentic systems
• Own the full lifecycle of evaluation data and collaborate cross-functionally to translate statistical uncertainty into product signals

Required Qualifications

• 10+ years of Machine Learning experience, focusing on Deep Neural Networks and model evaluation
• 1-2+ years of experience in post-training activities
• 1+ year experience creating benchmarks for evaluating LLMs
• Deep expertise in LLM-as-judge architectures and statistical rigor in experimental design
• Proven ability to manage the path from data collection to production deployment