Senior AI Quality Engineer

Job Summary

Owning the quality standards for AI initiatives, the full-time remote Senior AI Quality Engineer will design evaluation frameworks, build test harnesses, and define measurable quality criteria for LLM-driven and agentic workflows.

Key responsibilities:

• Design and maintain evaluation frameworks for LLM outputs and agentic workflows
• Build test harnesses to identify issues before production, including hallucinations and unsafe behaviors
• Integrate evaluation runs into CI/CD processes to ensure consistent quality checks for AI features

Required qualifications:

• 6+ years in QA, SDET, or test automation with real production experience
• Hands-on experience testing LLM-based or agentic systems
• Familiarity with modern automation stacks such as Playwright or Cypress
• Deep experience with test frameworks like vitest or pytest
• Working knowledge of CI/CD pipelines and API-first testing methodologies