Job Summary
A software development company has a current position open for a Remote Senior Spark Scala Data Engineer in Raleigh.
Individual must be able to fulfill the following responsibilities:
- Transform structured, semi-structured, and unstructured data for use by BI/ML/AI processes
- Build scalable data processing pipelines in Spark
- Write comprehensive unit and integration tests for all data processing
Qualifications for this position include:
- Competent with Scala, Spark, and the Spark Dataframe/Dataset API
- Scala (preferred), Java (meh, okay), Python (super cool, but let's elevate to Scala)
- ELT/ETL Best Practices
- Big Data mindset and experience
- Investigate and research new technologies
- Competent in basic SQL