Lead Automation QA Engineer

Job Summary

To validate AI products and ensure their reliability, the remote contract Lead Automation QA Engineer will design test plans, maintain coverage for AI outputs, and investigate failures while collaborating closely with engineering teams.

Key responsibilities

• Build and maintain test plans for AI products, focusing on input variants and deployment surfaces
• Run structured test passes across web UIs and APIs, providing actionable insights to engineers
• Maintain regression coverage and stay current on AI evaluation tooling and methodologies

Required qualifications

• 5+ years of manual and automation QA experience in product environments
• Hands-on experience testing AI/LLM-based products, including output validation and hallucination detection
• Strong test-planning skills for large combinatorial test matrices
• Proficiency in API testing tools such as Postman or Insomnia
• Basic SQL knowledge for verifying data integrity in databases