Lead Data Platform Engineer

Job Summary

Working remotely in a mid-senior level capacity, the full-time Lead Data Platform Engineer will manage end-to-end data pipelines, oversee AWS big data services, and lead the operational lifecycle for machine learning and generative AI infrastructure.

Key responsibilities

• Manage end-to-end data pipelines (ETL jobs) within agreed SLAs and oversee AWS core and big data services
• Lead Day 2 operational lifecycle for ML and GenAI infrastructure, focusing on high-availability production platforms and automated scaling
• Collaborate with product development and engineering teams to create AI-driven features while automating platform maintenance and standardizing infrastructure configurations

Required qualifications

• Bachelor's Degree and at least 8+ years of experience managing Big Data technologies and data pipelines
• 5+ years of experience in managing cloud infrastructure and platforms, such as AWS and Azure
• Strong expertise in MLOps and production-grade LLM operations, including managing high-availability model inference clusters
• Experience with containerization technologies like Amazon ECS and knowledge of Infrastructure as Code products such as Terraform
• Familiarity with the current engineering landscape in the generative AI space and a strong interest in AI technologies