Head of Evaluations

Job Summary

To ensure the highest standards of legal reasoning and accuracy, the remote Head of Evaluations will design and implement testing frameworks for AI products, focusing on legal benchmarks and collaborating with engineering teams.

Key responsibilities

• Design Legal Benchmarks for various legal tasks including contract drafting and legal research
• Audit AI Output by reviewing and scoring complex AI-generated legal texts for accuracy
• Define Evaluation Metrics to establish criteria for grading model performance

Required qualifications

• PhD or Masters in statistics, mathematics, machine learning, or equivalent
• Proven analytical skills in breaking down complex statutory frameworks
• Familiarity with Python, pandas, NumPy, Jupyter notebooks, and similar statistical models