Engineering Member - CUDA

Job Summary

A company is looking for a Member of Engineering (Pre-training / CUDA).

Key Responsibilities

• Profile large-scale training workloads and identify communication and computation bottlenecks
• Develop custom kernels to enhance training performance
• Collaborate with researchers to efficiently scale novel research ideas

Required Qualifications

• Strong engineering background with experience in CUDA and GPU programming
• Knowledge of distributed training and Large Language Models (LLMs)
• Proficiency in Python (PyTorch) and C/C++ programming
• Experience with profiling tools and algorithmic skills
• Familiarity with NVLink, NVSwitch, and NVShmem