DataOps Engineer

Job Summary

Seeking a full-time remote DataOps Engineer to manage and optimize the Data Marketplace platform, focusing on AI-driven observability, Azure infrastructure, and Databricks platform engineering while ensuring reliable data pipeline deployments and governance.

Key Responsibilities

• Implement AI-powered observability to detect anomalies and improve pipeline reliability
• Operate and maintain the full Azure data services stack and design ADF pipelines for data ingestion
• Own the deployment lifecycle of data workflows using GitHub Enterprise, ensuring rigorous CI/CD practices

Required Qualifications

• 7+ years of experience in DataOps, data engineering, or platform engineering in a production environment
• Expert-level experience with Databricks, including Delta Live Tables and Spark performance tuning
• Strong command of Azure data services such as ADF, ADLS Gen2, and Azure Monitor
• Demonstrated production use of AI tools in DataOps or observability workflows
• Proven CI/CD experience using GitHub Enterprise for data pipeline management