Principal Software Engineer

Job Summary

A company is looking for a Principal Software Engineer - Large-Scale LLM Memory and Storage Systems.

Key Responsibilities

• Design and evolve a unified memory layer for large-scale LLM inference
• Architect and implement integrations with leading LLM serving engines focusing on KV-cache management
• Co-design interfaces and protocols for efficient KV-cache sharing and storage across various systems

Required Qualifications

• Masters or PhD or equivalent experience
• 15+ years of experience in large-scale distributed systems or ML systems infrastructure
• Deep understanding of memory hierarchies and experience in multi-tier system design
• Experience with distributed caching or key-value systems optimized for low latency
• Hands-on experience with networked I/O technologies and familiarity with AI cluster deployments