Data Engineer

Job Summary

A company is looking for a Data Engineer to join their dynamic team.

Key Responsibilities

• Develop and maintain scalable data pipelines and systems using Scala, Python, Spark, and Databricks
• Design and implement unified data models with enterprise-wide consistency
• Optimize operations in Spark for handling large, unstructured, and nested datasets

Qualifications

• Extensive experience as a Data Engineer, focusing on Python programming in Spark/Databricks
• Proven expertise in designing and optimizing large-scale data pipelines, ETL processes, and data lakes
• Strong background in cloud platforms, ideally Azure, with hands-on experience in managing cloud-based data infrastructure
• Demonstrated ability to convert legacy systems into modern Spark/Databricks platforms
• Experience in troubleshooting and resolving complex data issues