PySpark Developer

Job Summary

To support data migration projects, the remote PySpark Developer will develop and optimize ETL pipelines using PySpark and Databricks, while assisting in the migration of legacy applications from Informatica to cloud-based solutions.

Key responsibilities:

• Develop and optimize ETL pipelines using PySpark and Databricks
• Assist in migrating legacy on-premise applications from Informatica to cloud-based solutions
• Cleanse and transform unstructured and structured data from multiple vendors

Required qualifications:

• Proficiency in PySpark with a focus on data processing and pipeline development
• Experience with Databricks workflows and cloud-based data environments
• Strong understanding of data cleansing and applying business rules
• Background in development, data modeling, or related fields
• Ability to work effectively in a small, collaborative team