Data Engineer

Job Summary

A company is looking for a Data Engineer with expertise in Databricks and real-time data streaming solutions.

Key Responsibilities

• Implement scalable and efficient PySpark code based on resource composition logic from the Product team
• Develop, optimize, and maintain PySpark applications in the Databricks environment
• Utilize and maintain CI/CD pipelines for automated build, testing, and deployment processes

Required Qualifications

• 5+ years of experience in data engineering roles
• Bachelor's degree in Computer Science, Engineering, or related field (or equivalent experience)
• Strong hands-on experience with PySpark and/or Apache Spark
• Experience with CI/CD pipelines and DevOps best practices
• Proficiency in GitHub for version control and code management