Data Engineering Lead

Job Summary

A company is looking for a Data Engineering Lead responsible for technical stewardship and designing scalable systems.

Key Responsibilities

• Architect and implement complex data pipelines using Azure Databricks and PySpark
• Optimize performance and cost of data processing while ensuring system integrity and reliability
• Lead a team towards cross-trained agility, managing sprint cycles and conducting code reviews

Required Qualifications

• 10+ years of experience in Data or Software Engineering with deep codebase involvement
• 3+ years as a Technical Lead managing agile teams
• Proven ability to lead high-impact teams while maintaining high individual output
• Experience with cross-training advocacy and scaling data processing through automation
• Core technical skills in Python, PostgreSQL, Azure Databricks, and PySpark