Remote Jobs Sign In

QA Automation Engineer

Get new jobs like this one by email. Create a free account for job alerts and to save the jobs you like.

Location: Remote
Compensation: To Be Discussed
Reviewed: Tue, Nov 17, 2026
This job expires in: 30 days

Job Summary

To support the growth of AI-driven products, the full-time QA Automation Engineer will build and maintain automated testing for AI voice and chat agents, validate LLM output quality, and ensure release quality in a fully remote environment.

Key responsibilities
  • Build and maintain automated testing for AI voice and chat agents, covering various conversational scenarios
  • Test and validate LLM output quality and retrieval relevance using evaluation harnesses and structured testing methods
  • Own release quality by implementing regression strategies and ensuring early detection of issues
Required qualifications
  • 5+ years as a QA Automation Engineer with a focus on testing AI systems
  • Proven experience in building testing frameworks and CI/CD-integrated automation from scratch
  • Hands-on experience testing conversational AI projects, including voice and chat agents
  • Strong understanding of LLMs, prompt engineering, and retrieval-augmented generation (RAG)
  • Proficiency in UI/E2E and API test automation, along with experience in database and performance testing

Complete Job Description

The complete job description is available to premium members. Premium membership includes:

Full access to 36,860 remote jobs from human-vetted companies, updated daily

Resume Builder - AI-powered tool to craft, enhance, and tailor your resume to a specific job

Twice-monthly live group coaching and the full Remote Career Center

20% member discount on Career Services

Backed by a 30-day money-back guarantee