Senior Databricks Data Engineer
Job Summary
Focused on ensuring data integrity, the full-time Senior Databricks Data Engineer will develop automated testing frameworks in PySpark and SQL to validate data pipelines against legacy systems, working remotely within the United States.
Key responsibilities
- Build and execute automated tests for data pipelines, ensuring data reconciliation and integrity across environments
- Define and implement data quality expectations within pipelines to prevent bad data from impacting operations
- Conduct user acceptance testing sessions with stakeholders, documenting results and tracking defects to resolution
Required qualifications
- 5+ years of experience in data engineering with a focus on building test and validation frameworks
- Proficient in PySpark and SQL for writing reconciliation and assertion logic
- Hands-on experience reconciling new platforms against legacy systems and explaining variances
- Strong understanding of incremental load correctness and data duplication resolution
- Experience with Python test frameworks and data testing tools
Complete Job Description
The complete job description is available to members. Premium membership includes:
Full access to 38,810 remote jobs from human-vetted companies, updated daily
Resume Builder - AI-powered tool to craft, enhance, and tailor your resume to a specific job
Twice-monthly live group coaching and the full Remote Career Center
20% member discount on Career Services
Backed by a 30-day money-back guarantee
Company Company Name
Headquarters Headquarters
Founded Founded
Website
The company description is available to members. Go Premium for full access to every job and company profile in our remote jobs database, backed by a 30-day money-back guarantee.
