LLM Evaluation Expert

Job Summary

A company is looking for a LLM Evaluation Expert specializing in Coding.

Key Responsibilities

• Critically analyze and evaluate code responses generated by LLMs across various programming languages
• Exercise expert judgment to select appropriate and efficient code solutions from multiple LLM-generated options
• Provide detailed feedback and collaborate with the AI research team to improve LLM's coding capabilities

Required Qualifications

• Advanced degree in Computer Science, Software Engineering, or a related field
• Extensive experience (5+ years) in software development across multiple programming languages
• Demonstrated ability to critically evaluate code quality and adherence to best practices
• Strong analytical and decision-making skills
• Experience in technical writing, particularly in creating coding examples or tutorials