Big Data Engineer

Job Summary

Seeking a hands-on Big Data Engineer, the full-time position will focus on developing and managing data pipelines using advanced Python and SQL skills while working collaboratively in a client-facing role.

Key responsibilities

• Develop and automate ETL processes, including scheduling jobs with tools like Airflow
• Optimize SQL queries for performance in data warehouse technologies, preferably Snowflake
• Troubleshoot data pipeline errors and implement data validation and quality checks

Required qualifications

• 10+ years of experience in data engineering
• Advanced proficiency in Python, including experience with multi-threading and functional programming
• Strong SQL skills, including complex joins and subqueries
• Experience with AWS services, particularly S3 for data storage and IAM for security
• Familiarity with version control systems, specifically GitHub, and CI/CD processes