Quality Analyst

Job Summary

Focused on evaluating AI-powered workflows, the full-time remote Quality Analyst II will assess AI output across the customer call lifecycle, document findings, and implement improvements within the internal AI platform, ORA.

Key Responsibilities

• Evaluate AI quality across the advisor stack by reviewing pre-call briefs, in-call guidance, and post-call outputs
• Iterate on existing AI workflows by refining prompts and updating knowledge base entries based on evaluation findings
• Surface trends and drive continuous improvement by maintaining logs of issues and recommending prioritized enhancements to leadership

Required Qualifications

• 2+ years of experience in Quality Assurance, Customer Support, AI evaluation, or a related role
• Working understanding of LLM behavior, prompt design, and conversational AI patterns
• General understanding of databases and data flow between systems
• Comfort reading code and making minor edits for debugging and integration purposes
• Bachelor's degree in a related field or equivalent practical experience