Research Engineer

Job Summary

To support the development of open models, the full-time Research Engineer will take architecture experiments through full pretraining runs and enhance the model and training stack, working remotely or onsite in San Francisco.

Key responsibilities

• Build and extend the pretraining code, transitioning small experiments into full training runs
• Evaluate model architecture changes against model quality and training cost
• Profile training jobs to improve throughput and ensure resilience during long runs with reliable checkpointing

Required qualifications

• Hands-on experience training language models and modifying training code
• Strong Python skills and experience with PyTorch or a comparable framework
• Practical understanding of model architectures and their optimization effects on convergence
• Ability to design and explain controlled ablation experiments
• Good judgment in determining when to investigate independently or collaborate with colleagues