AI QA Engineer

Job Summary

Experienced AI QA Engineers will find a full-time remote opportunity focused on testing AI/LLM products, APIs, and enterprise applications, ensuring quality through designing test plans, validating outputs, and maintaining regression coverage.

Key responsibilities

• Build and maintain test plans and golden-question test suites for AI products, ensuring comprehensive coverage against various input variants and user roles
• Run structured test passes, investigate failures, and validate evidence citations to ensure compliance with quality standards
• Maintain regression coverage as products evolve, partnering with engineers to verify acceptance criteria and staying current on AI evaluation tooling

Required qualifications

• 5+ years of manual and automation QA experience in production-grade environments
• Hands-on experience testing AI/LLM-based products, including output validation and hallucination detection
• Strong test-planning skills for large combinatorial test matrices
• Proficiency in API testing using tools like Postman or Insomnia and familiarity with OpenAPI specs
• Basic SQL knowledge for verifying data integrity in PostgreSQL or equivalent databases