Distributed Systems Engineer

Job Summary

Seeking a full-time remote Distributed Systems Engineer with expertise in Apache Spark internals to contribute production-grade code, debug and optimize core components, and influence architectural direction for performance and scalability.

Key responsibilities

• Contribute production-grade code to the Apache Spark project, focusing on Spark SQL and Structured Streaming
• Debug and optimize Spark internals including Catalyst, Tungsten, DAG Scheduler, Shuffle, and memory management
• Collaborate with cross-functional teams and mentor engineers through code reviews and design critiques

Required qualifications

• 6+ years of experience in software development
• Strong skills in Java and/or Scala
• Experience with distributed systems and concurrent or parallel programming
• Working knowledge of Spark internals (Catalyst, Tungsten, DAG Scheduler, Shuffle, or memory management)
• Familiarity with JVM performance characteristics (GC, memory, threading)