Performance Software Engineer

Job Summary

Focused on enhancing end-to-end large language model inference performance, the full-time remote Performance Software Engineer will optimize hardware-accelerated attention, accelerator tuning, and distributed KV cache management on custom AI accelerators.

Key Responsibilities

• Profile and optimize latency, throughput, and efficiency across various hardware and software components
• Develop and refine distributed inference capabilities, ensuring efficient data movement and KV cache management
• Build benchmarks and workload traces to identify and address performance bottlenecks in the system

Required Qualifications

• Strong proficiency in C++ and Python, with experience in profiling and optimizing complex systems using Linux VFIO interfaces
• Deep understanding of memory hierarchy, concurrency, and performance measurement techniques
• Hands-on experience with profilers, counters, and statistical methods for performance analysis
• Ability to debug across application, runtime, driver, firmware, and hardware layers
• Experience in bringing up software on new silicon and optimizing performance in production environments