Data Engineer

Job Summary

A company is looking for a Data Engineer to enhance its data ecosystem and support the open knowledge movement.

Key Responsibilities

• Design and build scalable data pipelines using tools like Airflow, Spark, and Kafka
• Monitor and implement systems for data quality and address potential issues
• Support data governance and lineage initiatives across data pipelines

Required Qualifications

• 3+ years of data engineering experience with on-premise systems (e.g., Spark, Hadoop)
• Hands-on experience in troubleshooting systems and pipelines for performance and scaling
• Proficient in Python or Java/Scala, with knowledge of SQL and various database/query dialects
• Experience with data pipeline tools like Airflow, Kafka, Spark, and Hive
• Familiarity with CI/CD processes and software containerization