Senior Staff Software Engineer

Job Summary

Leading the architecture and optimization of an AI inference platform, the full-time Senior Staff Software Engineer - AI/ML Inference Engineering will manage the end-to-end inference lifecycle, deploying optimized inference engines in production while working in a hybrid environment.

Key responsibilities

• Oversee the onboarding of cutting-edge models and deployment of inference engines across multiple backends on Kubernetes infrastructure
• Architect the intelligent gateway layer using Envoy and ext_proc to enhance access control and observability
• Drive the development of AI-powered work experiences by collaborating with cross-functional teams and ensuring operational excellence

Required qualifications

• 8+ years of backend software engineering experience with a focus on designing and operating distributed systems
• Expertise in Go or Python, object-oriented programming, and design patterns
• Experience with model-serving and runtime systems, such as vLLM or Triton Inference Server
• Bachelor's degree in computer science, software engineering, or a closely related technical field
• Proficiency in deploying and operating services in production Kubernetes environments, including CI/CD pipelines