Software Engineer III

Job Summary

Seeking a Software Engineer III to architect, develop, and maintain advanced automated data extraction systems in a full-time capacity, with the potential for remote work, focusing on overcoming technical barriers to ensure high-quality data flow into internal databases.

Key responsibilities:

• Design and deploy robust, distributed spiders and crawlers for data extraction from various web architectures
• Research and implement strategies to bypass anti-scraping technologies, including proxy rotation and CAPTCHA solving
• Create and optimize SQL schemas for large-scale data storage and perform complex data transformations

Required qualifications:

• Mastery of Python 3.x with experience in extraction frameworks such as Scrapy or Playwright
• Proven ability to bypass high-level bot detection technologies
• Expert-level SQL skills and experience with relational databases like PostgreSQL or MySQL
• Strong understanding of HTTP/S, TCP/IP, and browser-header manipulation
• Bachelor's degree in Computer Science or related field, with a minimum of 5 years of Software Engineering experience, including 2-3 years in web scraping or data extraction