QA Engineer

Job Summary

To validate AI products and ensure quality across various changes, the contract QA Engineer will work remotely, designing test plans, detecting hallucinations, and verifying citations while collaborating closely with engineering teams.

Key responsibilities

• Build and maintain test plans and golden-question suites for AI products, ensuring coverage against regressions and specific AI errors
• Run structured test passes on web UIs and APIs, investigating failures and validating evidence citations and role-based access
• Maintain regression coverage as products evolve, partnering with engineers on evaluation criteria and staying current on AI/LLM evaluation tooling

Required qualifications

• 5+ years of manual and automation QA experience in product environments
• Hands-on experience testing AI/LLM-based products, including output validation and hallucination detection
• Strong test-planning skills for large combinatorial test matrices
• Fluency in API testing tools such as Postman or Insomnia and familiarity with OpenAPI/Swagger specs
• Basic SQL knowledge for read-only queries to verify data states