Technical Lead Manager - Data Movement

Job Summary

Seeking a deeply hands-on Technical Lead Manager - Data Movement for a full-time hybrid role, responsible for designing and building a unified dataset read platform, defining dataset APIs, and ensuring reliability in data access for training infrastructure.

Key responsibilities

• Design and build a unified dataset read platform for multiple training frameworks
• Define dataset APIs, storage-format expectations, and migration paths for reproducible data access
• Build reliability into the data read path, including stateful iteration and caching mechanisms

Required qualifications

• Experience building or owning dataset, data loading, or distributed training infrastructure at scale
• Understanding of failure modes in large distributed training jobs and their impact on data systems
• Familiarity with stateful iterators, checkpoint/restart semantics, and high-throughput storage reads
• Proficiency in Python; experience with Rust or C++ is a plus
• Experience with multimodal, video, or reinforcement learning data pipelines