QA Engineer for AI Quality

Job Summary

To support a growing AI-driven greeting card platform, the part-time contract QA Engineer for AI Quality will manage the Playwright end-to-end testing suite, stabilize and extend coverage, and develop an evaluation system to improve AI output quality, all while working remotely with consistent EST overlap.

Key responsibilities

• Audit and stabilize the existing Playwright e2e suite, extending coverage for key user experiences
• Design and implement rubrics for evaluating AI-generated card outputs, ensuring quality across aesthetics and accuracy
• Provide concise reporting and actionable insights to the AI engineering and product teams based on evaluation findings

Required qualifications

• 3 to 5 years of experience in QA engineering with ownership of an automation framework
• Strong proficiency in Playwright, with the ability to read and extend unfamiliar test suites
• Hands-on experience in evaluating visual or multimodal AI outputs, particularly in aesthetics and composition
• Experience building and managing LLM evaluation pipelines, including rubric design and regression tracking
• Full stack literacy to effectively test modern web applications, including Next.js/React and Python/FastAPI