Washington State Licensed Agentic Judge

Job Summary

To support a tech real estate company, the part-time remote Washington State Licensed Agentic Judge will evaluate AI/ML processes related to verbal interactions in real estate, ensuring quality standards and providing feedback to optimize model performance.

Key responsibilities

• Review and evaluate annotated data to ensure it meets quality standards and project requirements
• Provide feedback and guidance to engineers to improve accuracy and optimize model performance
• Develop and maintain annotation guidelines and best practices for training data

Required qualifications

• Minimum of 3+ years of active experience as a licensed real estate agent or broker
• Demonstrated interest and experience in reviewing or evaluating AI-generated content
• Bachelor's degree in English, Linguistics, or a related field
• Experience with Natural Language Processing projects, focusing on transcriptions or localization
• Familiarity with tools and software used for Natural Language Processing annotation is a plus