Senior Data Engineer

Job Summary

To support data migration initiatives, the full-time Senior Data Engineer will build and optimize large-scale batch data pipelines in Databricks, utilizing Python, Spark, and advanced SQL, while collaborating with cross-functional teams in a remote or hybrid work environment.

Key responsibilities

• Build and migrate large-scale batch data pipelines from legacy platforms into Databricks
• Develop and optimize production-grade solutions using Python, Spark, and advanced SQL
• Lead technical design discussions and guide implementation within an established architecture

Required qualifications

• Recent hands-on experience building production batch solutions in Databricks within the past 12-18 months
• Strong development experience with Python and Apache Spark
• Advanced SQL skills, including complex query development and performance tuning
• Experience with large-scale migrations from legacy data platforms to cloud-based environments
• Familiarity with Azure data services such as Data Factory and Synapse Analytics is preferred