Data Engineer

Job Summary

A company is looking for a Data Engineer to enhance its data platform and support the open knowledge movement.

Key Responsibilities

• Design and build scalable data pipelines using tools like Airflow, Spark, and Kafka
• Implement monitoring systems for data quality to address potential issues promptly
• Support data governance and lineage by designing solutions to manage data across pipelines

Required Qualifications

• 3+ years of data engineering experience with on-premise systems (e.g., Spark, Hadoop)
• Hands-on experience in troubleshooting systems and pipelines for performance and scaling
• Proficient in Python or Java/Scala, with knowledge of SQL and various database/query dialects
• Familiarity with data pipeline tools like Airflow, Kafka, Spark, and Hive
• Understanding of engineering best practices with a focus on maintainable and reliable code