GPU Inference Engineer
Get new jobs like this one by email. Create a free account for job alerts and to save the jobs you like.
Location: Remote
Compensation: Salary
Reviewed: Thu, Nov 19, 2026
This job expires in: 30 days
Job Summary
Joining a dynamic team, the full-time GPU Inference Engineer will own and enhance the GPU serving stack using vLLM and SGLang, focusing on improving evaluation throughput and synthetic data generation in a hybrid work environment.
Key responsibilities
- Own and improve the GPU serving stack, including configuration and scheduling choices
- Speed up large-scale evaluation and synthetic data generation while designing inference pipelines
- Profile and benchmark the inference engine behind the API service, adapting the API layer as needed
Required qualifications
- Hands-on experience with LLM inference on GPUs using frameworks like vLLM or SGLang
- First-principles understanding of LLM inference, including transformer forward pass and latency trade-offs
- Solid grasp of GPU fundamentals related to inference without needing CUDA kernel experience
- Experience in profiling and benchmarking inference workloads
- Strong proficiency in Python and PyTorch with the ability to navigate large open-source codebases
Complete Job Description
The complete job description is available to premium members. Premium membership includes:
Full access to 38,909 remote jobs from human-vetted companies, updated daily
Resume Builder - AI-powered tool to craft, enhance, and tailor your resume to a specific job
Twice-monthly live group coaching and the full Remote Career Center
20% member discount on Career Services
Backed by a 30-day money-back guarantee