Licensed Physician AI Evaluator

Job Summary

To evaluate clinical EHR vignettes for a medical AI evaluation program, a part-time Licensed Physician AI Evaluator will engage in project-based tasks remotely, focusing on scoring clinical reasoning quality across various medical systems.

Key responsibilities

• Evaluate clinical EHR vignettes for accuracy, completeness, and reasoning quality across multiple medical systems
• Score benchmark items based on vignette accuracy, answer correctness, and reasoning traces, providing short rationales
• Participate in independent blind reads and physician adjudication to enhance AI tools for clinical use

Required qualifications

• Medical degree (MD or DO) with an active, unrestricted US medical license
• Board certification or completed residency training with recent direct patient care experience
• Broad diagnostic and treatment experience in internal, family, emergency, or hospital medicine
• Strong clinical reasoning skills in differential diagnosis and evidence-based guidelines
• Prior experience in medical AI evaluation or clinical content review is a plus