Senior Architect, Model Performance

Job Summary

Driving the performance modeling and post-training of large language models, the full-time Senior Architect, Model Performance will develop frameworks for measuring coding model performance, optimize post-training pipelines, and collaborate with cross-functional teams to enhance model capabilities, primarily in a remote setting.

Key responsibilities

• Develop frameworks to measure and analyze coding model performance across various software engineering tasks
• Optimize post-training pipelines, including supervised fine-tuning and reinforcement learning strategies
• Analyze benchmark results and model behaviors to identify opportunities for improvement in coding tasks

Required qualifications

• Bachelor's degree (or equivalent experience) in Computer Science or AI with 12+ years of relevant work experience
• Excellent programming skills in Python and a strong understanding of large language models and transformer architectures
• Foundational knowledge of reinforcement learning algorithms and implementation
• Deep understanding of software engineering fundamentals and automated testing
• Proven track record in leading model development life cycles, including dataset versioning and experiment tracking