Applied Research Scientist, LLM Evaluation

Job Summary

Leading research and experimentation in evaluation design and model improvement, the full-time remote Applied Research Scientist, LLM Evaluation & Post-Training will focus on developing evaluation frameworks and methodologies for LLM and multimodal systems while collaborating with cross-functional teams.

Key responsibilities

• Define and execute a research agenda centered on LLM evaluation and post-training methodologies
• Design rigorous experiments to assess the impact of evaluation methodologies on model fine-tuning and outcomes
• Collaborate with AI/ML Research Engineers and Language Data Scientists to translate research insights into scalable evaluation pipelines

Required qualifications

• MS/PhD in Computer Science, Machine Learning, Statistics, Applied Mathematics, AI, or a related quantitative scientific field (PhD strongly preferred)
• 5+ years of relevant experience in applied research or research science in ML/AI, particularly with LLMs or foundation models
• Demonstrated experience in LLM evaluation, benchmarking, and post-training research
• Strong coding skills in Python for research experimentation and analysis
• Experience with modern ML tooling/frameworks such as PyTorch or TensorFlow for designing and executing experiments