Senior Azure Databricks Engineer

Job Summary

To lead the design and delivery of modern enterprise data platforms, the full-time Senior Azure Databricks Engineer will own the architecture and implementation of lakehouse platforms on Databricks and Azure, ensuring governance, infrastructure, and production operations while working remotely.

Key responsibilities

• Lead end-to-end architecture for enterprise lakehouse platforms, making design decisions across storage, compute, governance, and security
• Design and implement Unity Catalog strategies aligned with client security and compliance requirements
• Architect and build scalable batch and streaming pipelines using Python, PySpark, and SQL with Delta Lake

Required qualifications

• 7+ years of professional experience in data engineering or software engineering, with at least 3 years of hands-on Databricks experience
• Expert-level proficiency in Python, PySpark, and SQL, with experience in automated testing using PyTest
• Deep experience with Delta Lake, Unity Catalog, and Terraform for provisioning Azure and Databricks infrastructure
• Strong Azure experience, particularly with ADLS Gen2 and Microsoft Entra ID
• Proven experience building and maintaining CI/CD pipelines and Git-based development workflows