Data Collection Engineer

Job Summary

To design, scale, and maintain distributed web scraping and data extraction infrastructure, the full-time remote Data Collection Engineer will build resilient data pipelines, ensure data quality through automated validation, and manage containerized workloads at scale.

Key responsibilities

• Design and deploy high-performance, distributed web scrapers using Python and Scrapy for efficient data extraction
• Utilize browser scripting tools to extract data from dynamic and JavaScript-heavy websites
• Deploy and manage scraping workloads on Kubernetes, ensuring optimal resource allocation and fault tolerance

Required qualifications

• 7+ years of professional software engineering experience focused on web scraping, data engineering, or distributed systems
• Expert-level proficiency in Python and deep experience with Scrapy and distributed scraping architectures
• Proven experience with browser automation tools such as Playwright, Selenium, or Puppeteer
• Hands-on experience with Elasticsearch for indexing and querying data
• Bachelor's degree in Computer Science or Engineering