Senior Databricks Data Engineer

Job Summary

Focused on building robust data ingestion frameworks, the full-time Senior Databricks Data Engineer will establish an S3 landing zone and ingestion pipelines for engineering data, ensuring resilience and data quality in a fully remote environment.

Key responsibilities

• Establish the S3 landing zone and ingestion framework for file-based engineering sources
• Build Auto Loader pipelines for incremental ingestion of data from Cameo, Teamcenter, and Capital exports
• Implement data quality checks and manage schema evolution in Databricks pipelines

Required qualifications

• 6+ years of experience in data engineering, specifically with Databricks ingestion pipelines
• Deep knowledge of Auto Loader and incremental file ingestion from S3
• Experience parsing structured and semi-structured data formats (CSV, XML, JSON)
• Proficiency in PySpark and SQL, with a solid understanding of Delta Lake
• Familiarity with PLM and MBSE vocabulary for effective communication with engineering SMEs