Senior Data Engineer

Job Summary

Seeking a highly skilled Senior Data Engineer, the full-time remote position will focus on building, optimizing, and maintaining robust data pipelines for real-time analytics and AI/ML initiatives while collaborating with U.S.-based teams during core hours.

Key responsibilities

• Build and maintain complex data pipelines using AWS Glue, Step Functions, or Databricks Workflows
• Develop and deploy real-time ingestion pipelines utilizing AWS Kinesis or Kafka
• Own end-to-end data validation and QA by implementing automated data quality checks within ETL/ELT pipelines

Required qualifications

• 6-8+ years of experience in data engineering with a focus on large-scale distributed systems
• Expert-level proficiency in Python and PySpark, along with strong SQL skills
• Deep hands-on experience with Snowflake or Databricks in an AWS ecosystem
• Proven track record in building streaming applications using Kinesis or Kafka
• Demonstrated experience in implementing automated testing frameworks and data profiling