Annotation Data Engineer

Job Summary

To enhance the quality of machine learning data operations, the full-time remote Annotation Data Engineer will define and maintain annotation quality standards, develop metrics for performance measurement, and analyze quality trends in collaboration with cross-functional teams.

Key responsibilities

• Define and maintain annotation quality standards, guidelines, and acceptance criteria
• Develop metrics and reporting for accuracy, consistency, and reviewer agreement
• Analyze quality trends and coordinate corrective actions with annotation operations and engineering teams

Required qualifications

• Experience in annotation quality, machine learning data operations, or quality engineering
• Proficiency in developing quality metrics and performance measurement systems
• Experience with large-scale annotation programs and evaluating generated outputs
• Working knowledge of SQL, Python, and data analysis tools
• Ability to define annotation instructions, sampling plans, and audit processes