Data Pipeline Engineer

Job Summary

Building and operating the data backbone of a large-scale data-platform, the full-time remote Data Pipeline Engineer will focus on bulk and streaming ingestion, medallion-layered storage, and data quality reconciliation in a HIPAA/SOX-regulated environment.

Key responsibilities

• Develop batch-seed and event-tail ingestion processes, ensuring idempotent upserts and deduplication
• Operate medallion layers with pipeline orchestration, including checkpoints and full observability
• Implement data-quality gates and reconciliation engines to ensure data integrity before publication

Required qualifications

• 5+ years of experience building production data pipelines at scale
• Proficiency with Kafka, including consumers, producers, and idempotent processing patterns
• Strong SQL skills and solid understanding of ETL fundamentals
• Experience with Java and/or Python in production environments
• Familiarity with data-quality frameworks and entity resolution techniques