Principal AI Evaluation Strategist

Job Summary

Serving as a consultative expert, the full-time Principal AI Data & Evaluation Strategist will provide strategic guidance during project discovery, solution design, and evaluation framework creation in a remote, flexible engagement focused on frontier AI work.

Key responsibilities

• Lead project discovery, translating business goals into AI data requirements and defining project success metrics
• Design training and evaluation datasets, establish ground-truth standards, and identify coverage gaps and edge cases
• Develop human-evaluation frameworks and quality-measurement methodologies while challenging assumptions that may impact project quality

Required qualifications

• 10+ years in AI, data science, machine learning, research, or AI operations
• Experience designing large-scale AI data programs
• Experience with LLM evaluation, human feedback systems, AI benchmarking, and dataset development
• Strong written English and the ability to communicate complex ideas clearly
• Comfortable working independently in a remote environment