Databricks Platform Engineer

Job Summary

Seeking a full-time remote Databricks Platform Engineer to implement and operationalize core platform capabilities on Databricks running on AWS, focusing on frameworks, automation, observability, and reliable delivery aligned to established platform standards.

Key Responsibilities

• Implement metadata-driven ingestion frameworks and standard ingestion patterns while building batch, incremental, and CDC ingestion pipelines
• Establish pipeline monitoring, metrics, and alerting, supporting operational validation and failure simulation
• Build CI/CD pipelines using GitHub Actions, ensuring configuration validation and secret management via AWS Secrets Manager

Required Qualifications

• 5-8 years of data or platform engineering experience
• 3+ years of hands-on experience with Databricks
• Strong proficiency in PySpark and SQL
• Experience with Delta Lake and Unity Catalog
• Familiarity with Git-based development