Senior Inference Engineer

Job Summary

Partnering directly with the CTO, the full-time Senior Inference Engineer will build and optimize the platform's inference layer, deploying models to production and managing the entire query-to-response pipeline in a remote environment.

Key responsibilities

• Deploying models to production and owning the full pipeline from query to response
• Building and operating serving infrastructure using frameworks like vLLM, SGLang, or TensorRT-LLM
• Leading the engineering direction for inference, collaborating with Product to shape future developments

Required qualifications

• Experience deploying and serving LLMs using frameworks such as vLLM, SGLang, or TensorRT-LLM
• Hands-on knowledge of inference optimization techniques like quantization, batching, caching, and routing
• Strong programming skills in Python or Golang with real production code experience
• Ability to communicate technical concepts clearly to both technical and non-technical stakeholders
• Familiarity with containerized environments (Docker, Kubernetes) is a plus