AWS Lakehouse Data Engineer

Job Summary

To design and operate a cloud-native data platform, the full-time AWS Lakehouse Data Engineer will build a modern lakehouse on Amazon S3 using AWS-native services, focusing on developing scalable batch and streaming ingestion, ETL/ELT pipelines, and ensuring data governance and quality in a remote environment.

Key responsibilities

• Design and implement batch and streaming data ingestion pipelines from various sources, optimizing ETL/ELT processes using Python and PySpark
• Deliver a scalable AWS-native lakehouse data platform, managing data reliability, performance optimization, and cost efficiency
• Implement data governance and access control measures, ensuring compliance and operational quality through automated checks and metadata management

Required qualifications

• Bachelor's degree in Engineering, Information Technology, Computer Science, Data Engineering, or a related field, or four years of equivalent practical experience
• Six years of relevant experience in data engineering and AWS-native architectures, particularly with Amazon S3 and related services
• Hands-on experience developing production ETL/ELT pipelines using Python and PySpark
• Strong knowledge of Apache Iceberg, including its features like ACID transactions and schema evolution
• Experience with metadata management and governance capabilities, including AWS Lake Formation and IAM