Senior Data Infrastructure Engineer
Job Summary
Specializing in web scraping and large-scale data ingestion, the full-time remote Senior Data Infrastructure Engineer will build and maintain a core ingestion engine for social media data, overcoming complex platform defenses to deliver millions of records daily.
Key responsibilities
- Build and manage stealth scraping clusters using residential proxy networks and browser automation tools
- Develop fault-tolerant, scalable web-scraping pipelines for high-volume data extraction from various platforms
- Design distributed queues and workflow engines to manage asynchronous scraping tasks efficiently
Required qualifications
- 4+ years of experience in high-volume web scraping, data engineering, or reverse engineering
- Deep experience with anti-bot evasion techniques and tools
- Proficiency in Python or Go, with knowledge of web scraping frameworks like Playwright or Puppeteer
- Experience with distributed systems and task queues such as Temporal or Kafka
- Strong SQL skills and familiarity with analytical data lakes or warehouses
Complete Job Description
The complete job description is available to members. Premium membership includes:
Full access to 42,491 remote jobs from human-vetted companies, updated daily
Resume Builder - AI-powered tool to craft, enhance, and tailor your resume to a specific job
Twice-monthly live group coaching and the full Remote Career Center
20% member discount on Career Services
Backed by a 30-day money-back guarantee
Company Company Name
Headquarters Headquarters
Founded Founded
Website
The company description is available to members. Go Premium for full access to every job and company profile in our remote jobs database, backed by a 30-day money-back guarantee.
