Lead Data Engineer

Job Summary

A company is looking for a Lead Data Engineer.

Key Responsibilities:

• Develop and maintain data pipelines, ELT processes, and workflow orchestration using Apache Airflow, Python, and PySpark
• Design and implement custom connectors for data ingestion from diverse sources
• Monitor and troubleshoot data pipelines, ensuring accuracy and compliance of data

Required Qualifications:

• Bachelor's degree in Computer Science, Engineering, or a related field
• Experience in data engineering, ELT development, and data modeling
• Proficiency in Apache Airflow and Spark for data transformation and management
• Strong understanding of SQL and database concepts
• Experience implementing DataOps principles and CI/CD pipelines