AI QA Engineer

Job Summary

Experienced AI QA Engineers will find a full-time remote opportunity focused on testing AI/LLM products, including designing test plans, validating outputs, and maintaining regression coverage.

Key responsibilities

• Build and maintain test plans and golden-question suites for AI products, addressing various input variants and content states
• Run structured test passes across web UIs and APIs, investigating failures and validating evidence citations
• Ensure regression coverage as products evolve, partnering with engineers on evaluation rubrics and acceptance criteria

Required qualifications

• 5+ years of manual and automation QA experience in product environments
• Hands-on experience testing AI/LLM-based products, including hallucination detection and edge-case coverage
• Strong test-planning skills for large combinatorial test matrices
• Proficiency in API testing tools such as Postman or Insomnia and familiarity with OpenAPI/Swagger specs
• Basic SQL skills for read-only queries to verify data state in PostgreSQL or equivalent