Principal Distributed Systems Engineer
Location: Remote
Compensation: Base+commission
Reviewed: Thu, Sep 03, 2026
This job expires in: 30 days
Job Summary
Owning the technical vision and architecture for distributed tracing within Workday's Observability Platform, the full-time remote Principal Distributed Systems Engineer will design and build a multi-petabyte, low-latency tracing infrastructure while mentoring engineers and setting technical direction across teams.
Key responsibilities
- Architect and build Workday's distributed tracing platform on ClickHouse/Tempo for high-scale ingestion and query performance
- Own the big-data pipeline for tracing data, including ingestion, processing, and storage management
- Drive performance and scaling optimizations across ingestion and query paths for the tracing services
Required qualifications
- 14+ years of experience in software development engineering
- 6+ years of experience in designing and operating complex distributed system architectures
- 8+ years of experience with at least two programming languages such as Java, Python, or Go
- Bachelor's degree in Computer Science, Engineering, or a related field; a Master's degree is preferred
- Expert-level ability in Algorithmic Thinking and deep expertise in API Development and Distributed Systems Software principles
Complete Job Description
The complete job description is available to members. Premium membership includes:
Full access to 44,891 remote jobs from human-vetted companies, updated daily
Resume Builder - AI-powered tool to craft, enhance, and tailor your resume to a specific job
Twice-monthly live group coaching and the full Remote Career Center
20% member discount on Career Services
Backed by a 30-day money-back guarantee