Automation Test Engineer

Job Summary

Seeking an experienced Automation Test Engineer, the full-time remote position will design and maintain automated test frameworks for AI/ML platforms, develop model validation pipelines, and collaborate with engineers to ensure quality in software releases.

Key responsibilities

• Support the development of an AI/ML platform for the Centers for Medicare and Medicaid Services (CMS)
• Design and implement scalable automated test frameworks for web apps, APIs, and AI-driven features
• Define and execute comprehensive test plans, including functional, regression, and AI-specific testing strategies

Required qualifications

• 5+ years of experience in software QA/automation engineering, with at least 1 year focused on AI/ML product testing
• Strong proficiency in an automation framework (e.g., Selenium, Cypress) and a scripting language (Python preferred)
• Hands-on experience with testing AI models, including validation metrics and dataset versioning
• Familiarity with AI observability tools like MLflow or TensorBoard
• Experience in Agile DevSecOps environments with modern CI/CD tooling (e.g., Jenkins, GitHub Actions)