Lead Data Consultant

Job Summary

To support a client engagement focused on modernizing an Azure Synapse data platform, the full-time Lead Data Consultant will work remotely to refine Microsoft Fabric architecture, develop scalable data pipelines, and implement automated CI/CD workflows.

Key responsibilities

• Evaluate and refine the existing Microsoft Fabric architecture and design a metadata-driven ingestion framework
• Migrate prioritized data sources and workloads from Azure Synapse into Microsoft Fabric while ensuring migration accuracy through validation processes
• Collaborate with client engineers to establish sustainable engineering standards and contribute to architectural decisions

Required qualifications

• 5+ years of experience with Azure Data Services, including Microsoft Fabric and Azure Synapse
• Hands-on experience in production environments with Microsoft Fabric, including lakehouses and pipelines
• Strong experience in building and maintaining production ETL/ELT pipelines and metadata-driven ingestion frameworks
• Proficiency in developing Spark notebooks and data transformation workflows using PySpark and Spark SQL
• Experience implementing Git-based CI/CD workflows for data platform artifacts using tools like GitHub Actions or Azure DevOps