Senior Databricks Data Engineer

Job Summary

Building robust ingestion frameworks, the full-time Senior Databricks Data Engineer will establish S3 landing zones and develop resilient pipelines for engineering data exports, working remotely within the United States.

Key responsibilities

• Establish the S3 landing zone and ingestion framework for file-based engineering sources
• Build Auto Loader pipelines for incremental ingestion of data from Cameo, Teamcenter, and Capital
• Implement data quality checks and manage schema evolution in Databricks pipelines

Required qualifications

• 6+ years of data engineering experience with a focus on Databricks ingestion pipelines
• Deep knowledge of Auto Loader and incremental file ingestion from S3
• Experience parsing structured and semi-structured data formats (CSV, XML, JSON)
• Proficient in PySpark and SQL, with understanding of Delta Lake internals
• Fluent in PLM and MBSE terminology for effective communication with engineering SMEs