Data Engineer

Job Summary

Owning the real-time data infrastructure, the part-time Data Engineer I will manage streaming pipelines, troubleshoot incidents, and ensure data integrity while working remotely.

Key responsibilities

• Oversee end-to-end operations of real-time data pipelines for risk assessment using Vertex AI and Gemini
• Manage Pub/Sub infrastructure, including message delivery and incident response
• Scale, monitor, and optimize Dataflow jobs processing petabytes of data, diagnosing failures and implementing fixes

Required qualifications

• Hands-on experience building or operating streaming data pipelines (Pub/Sub, Kafka, or similar)
• Proven ability to debug and optimize GCP Dataflow or comparable Beam-based stream processing
• Strong understanding of SLAs, observability, and incident response
• Proficiency in Python, SQL, and Linux
• Experience integrating Vertex AI / Gemini or similar LLM platforms into data workflows