Lead Prompt Engineer

Job Summary

A company is looking for a Lead Prompt Engineer to guide and manage a team through the technical migration process of templates to LLM autoraters.

Key Responsibilities

• Utilize Automatic Prompt Generation (APG) tools to create baseline prompts for complex template clusters
• Supervise Automated Prompt Optimization (APO) tool outputs and address deadlocks or plateaus
• Monitor shadowbot runs and measure autorater quality against human crowd baselines

Required Qualifications

• Master's or Doctorate degree in Computer Science, Data Science, Computational Linguistics, Human-Computer Interaction, Cognitive Science, or a related field
• At least 7 years of experience as a Prompt Engineer with proven expertise in tuning Large Language Models (LLMs)
• Strong proficiency in data analysis and using SQL or other analytics tools
• Ability to quickly learn and master proprietary tools with minimal supervision
• Native fluency in English and must be based in the United States