Senior Azure Databricks Engineer

Job Summary

To lead the design and delivery of modern enterprise data platforms, the full-time Senior Azure Databricks Engineer will own the architecture and implementation of lakehouse platforms on Databricks and Azure, focusing on governance, infrastructure, and production operations in a fully remote role.

Key responsibilities

• Lead end-to-end architecture for enterprise lakehouse platforms, making design decisions across storage, compute, governance, and security
• Design and implement Unity Catalog strategies, ensuring compliance with client security requirements
• Architect and build scalable batch and streaming pipelines using Python, PySpark, and SQL

Required qualifications

• 7+ years of experience in data engineering or software engineering, with at least 3 years of hands-on Databricks experience
• Expert-level proficiency in Python, PySpark, and SQL, with experience in automated testing using PyTest
• Deep experience with Delta Lake, Unity Catalog, and Terraform for provisioning Azure resources
• Strong knowledge of Azure services, particularly ADLS Gen2 and Microsoft Entra ID
• Proven experience in building and maintaining CI/CD pipelines for data platforms