Senior Data Engineer

Job Summary

To support the development of a cutting-edge retrieval platform, the full-time Senior Data Engineer will build and manage the Bedrock Knowledge Base and OpenSearch Serverless collection, implement hybrid retrieval systems, and develop the retrieval API, all while working remotely.

Key responsibilities

• Stand up the Bedrock Knowledge Base and OpenSearch Serverless vector collection from infrastructure code
• Implement hybrid retrieval systems with metadata filtering and tune scoring parameters for optimal performance
• Build the retrieval API on API Gateway and Lambda, ensuring proper authentication and entitlement enforcement

Required qualifications

• 4+ years as a software or ML engineer with experience delivering RAG or enterprise search systems to production
• Hands-on experience with Amazon Bedrock Knowledge Bases and OpenSearch Serverless vector search
• Proficiency in Python and experience with Lambda-based APIs behind API Gateway
• Familiarity with infrastructure-as-code tools like Terraform or AWS CDK for deployment
• Understanding of chunking, embedding model selection, and the trade-offs between relevance, latency, and cost