Somali AI Evaluator

Job Summary

To support an upcoming project, the contracted Somali AI Evaluator will test and evaluate AI chatbots and language models remotely, providing critical insights to enhance response quality through direct user interaction.

Key responsibilities

• Test and evaluate AI chatbots through structured conversations using assigned goals and prompts
• Engage in voice-based interactions with AI models, submitting recorded sessions and evaluations as specified
• Deliver feedback and insights in the required formats, such as write-ups, ratings, screenshots, or recordings

Required qualifications

• Native or expert-level fluency in Somali, as spoken in Somalia
• Strong English reading, writing, and communication skills
• Hands-on experience using AI models or large language models
• Access to a smartphone, computer, and reliable internet, with comfort in using web-based tools
• Able to pass a language proficiency assessment and AI evaluation task