Data Analysis and Evaluation Specialist

Job Summary

A company is looking for a Member of Technical Staff, Data Analysis and Evaluation.

Key Responsibilities

• Design and oversee data collection tasks, ensuring data quality and supporting human annotators
• Develop and apply statistical methods to evaluate dataset quality and reliability
• Analyse the generalisability and robustness of machine learning systems across diverse use cases

Required Qualifications

• Strong expertise in designing data collection tasks and working with human annotators
• Proficient in statistical methods and evaluating scientific experiments related to data collection
• Experience in analysing datasets for quality, biases, and suitability for training machine learning models
• Hands-on experience with training large language models on distributed infrastructures
• Familiarity with programming languages such as Python and ML frameworks like PyTorch or TensorFlow