Principal Research Scientist

Job Summary

Owning the research agenda for evaluating AI agents, the full-time Principal Research Scientist will design methodologies, contribute to product evaluations, and mentor teams while working remotely.

Key responsibilities

• Set and maintain evaluation methodologies for AI agents, ensuring they are defensible and current
• Collaborate with engineers and product managers to integrate evaluation methods into existing products
• Mentor researchers and engineers, guiding them in evaluation design and maintaining high-quality standards

Required qualifications

• 5+ years of experience in building or evaluating machine learning systems, particularly with LLMs
• A graduate degree (MS or PhD) in machine learning, NLP, computer science, statistics, or a related field
• A proven track record of original research or significant improvements in machine learning, demonstrated through publications or projects
• Experience managing small to medium teams, with a focus on mentoring and technical direction
• Strong understanding of statistical methodologies and experimental design in the context of AI evaluation