Machine Learning Engineer

Job Summary

A company is looking for a Machine Learning Engineer specializing in LLM Evaluation & Automation.

Key Responsibilities:

• Design and implement automated systems and pipelines for evaluating LLM outputs
• Develop metrics and KPIs to measure output quality, accuracy, and consistency using LLM-based evaluations
• Collaborate with Engineering teams to create automated logic checks and validation tools

Required Qualifications:

• 5+ years of experience in ML engineering, NLP, or AI/ML automation
• Advanced degree (MS/PhD) in Statistics, Data Science, Computational Social Science, Quantitative Psychology, or a related field
• Hands-on experience in prompt engineering and designing LLM-based evaluation systems is preferred
• Strong understanding of machine learning principles with a focus on NLP and advanced LLM capabilities
• Expertise in building automated evaluation or QA pipelines