Senior AI Engineer

Job Summary

Seeking a highly skilled Senior AI Engineer for a full-time remote position, responsible for designing and managing ClickUp's AI evaluation systems, building core agent harnesses, and developing self-improvement loops to enhance AI performance and reliability.

Key Responsibilities:

• Design, build, and own ClickUp's AI evaluation system, including golden datasets and automated scoring
• Build and evolve the core agent harness, collaborating with product and engineering teams
• Architect scalable AI platform services and maintain robust APIs for integrating AI features

Required Qualifications:

• 5+ years of software engineering experience, with 2+ years in production LLM or ML systems
• Hands-on experience building LLM evaluation systems tied to real product quality
• Strong experiment design and statistical rigor, including A/B testing methodologies
• Proven experience integrating and managing multiple LLMs with a focus on quality and performance
• Deep expertise in backend engineering and cloud-native technologies such as Kubernetes and AWS