Big Data Engineer

Job Summary

Seeking a mid-level Big Data Engineer for a remote contract position, the candidate will develop, implement, and test Big Data applications on the AWS Cloud, utilizing frameworks such as Spark and Hadoop while leveraging their experience in Java, Scala, and Python.

Key responsibilities:

• Develop, implement, and test Big Data applications (API & Batch) on the AWS Cloud
• Conduct performance tuning for Batch Applications, including Spark, Hadoop, EMR, Java, and Python
• Create automation pipelines for data ingestion and transformation using Spark with Snowflake

Required qualifications:

• 7+ years of experience in analysis, coding, implementation, and testing of Big Data applications on the AWS Cloud
• Proficiency in Java, Scala, Kotlin, Cassandra, Python, Hadoop, HDFS, Spark, and the MapReduce framework
• Coding experience with Scala, PySpark, and Java for REST API and SOAP web services
• Experience with AWS services including EMR, EC2, ECS, S3, Airflow, Step Functions, and AWS API Gateway
• Linux experience, including shell scripting and Python