Web Scraping Architect

Fred Lavoie, President & Co-Founder of Deck

At Deck, we believe data is to business what water is to life – vital and indispensable.

Our Values

Resilient Mindset

Comfort breathes complacency – hard times make great improvements. We embrace challenges and laugh in the face of adversity.

Purpose-Driven Impact

Help us make credential-protected data universally consumable. Every source turns into actionable insights that empower businesses to avoid “data dehydration.”

Join seasoned founders with a clear industry vision and track record. We're not just building a company - we’re setting a new standard in data automation.

Elite Team Culture

Deck operates like an elite sports team: hard work meets purpose, and every contribution shapes our future. Here, belonging is earned and valued. Cohesion and alignment will win the day.

Accelerated Growth

If you thrive on speed and innovation, you’ll find a natural home with us. At Deck, failure is just a step toward success. Without failures, there ain’t no learning.

We seek people who are genuinely excited about joining Deck and resonate with our purpose. Passion for our journey and shared goals is essential.

Grit and Independence

At Deck, we’re a fast-paced, high-expectation startup. We value self-starters who can think on their feet, adapt quickly, and seize opportunities as they arise.

Curiosity and Coachability

We thrive on growth and learning. A receptive mindset and openness to giving and receiving feedback are vital as we push boundaries and refine our work.

Love for Complexity

Deck is pioneering new ground in data accessibility. The journey will involve challenges, and we need team members who are motivated by problem-solving and resilience.

Sound Judgment

While we embrace diverse perspectives to make the best decisions, we expect commitment once decisions are made. We need team players who can align and move forward together.

Get Ready for an Engaging, Thought-Provoking, and Challenging Interview

Discover our culture and showcase your true potential.

Experience Our Culture Through Real-Life Questions

We want you to gain a true sense of our culture and style. Expect specific, experience-based questions (like "Tell me about a time when you...") to explore your background.

Exploring Your Experience and Resilience

We'll ask you to complete questionnaires to understand the full scope of your experience and may introduce scenarios outside your comfort zone to assess your resilience and alignment with our values.

An Engaging, Respectful, and Flexible Interview

Prepare to be challenged - professionally and with a touch of humor. You’ll be respected throughout and part of an empowering, engaging process. We're flexible with scheduling to accommodate your needs and make the experience as seamless as possible.

Open Roles

Check out our in-office and fully flexible roles.

Web Scraping Architect

Location

Full time

Location Type

Hybrid

Department

About Deck

Deck is building the data infrastructure for the internet. We make scattered, login-protected data instantly accessible through clean APIs and integrations—empowering businesses to act fast and smart, with no friction.

We’re a team of builders from top-tier tech companies who believe one thing: great ideas need great data. If you thrive in early-stage chaos, own your work like a founder, and think in frameworks—not features—Deck might be your next home.

Role

We’re looking for a skilled Web Scraping Architect to design and implement scalable data extraction solutions. In this role, you’ll develop efficient scraping systems, navigate authentication challenges, ensure compliance with legal standards, and collaborate on API integrations to enhance data accessibility. Ensure Deck’s ability to connect to data providers despite their anti-bots efforts.

Responsibilities

Advanced Web Scraping & Anti-Bot Evasion

• Design and maintain robust data extraction pipelines to bypass anti-scraping measures used by utility providers.
• Develop adaptive request timing, session persistence, and fingerprint randomization techniques.
• Utilize CAPTCHA-solving techniques, including AI-based solvers, third-party services, and human-in-the-loop solutions.
• Build high-throughput scraping clusters using asynchronous networking (e.g., Twisted, asyncio, Scrapy, Celery).
• Implement distributed scraping architectures leveraging cloud computing (AWS, GCP, Azure) and containerization (Docker, Kubernetes).
• Optimize scrapers for low latency, high fault tolerance, and rapid scale-up/down capabilities.
• Monitor and handle large-scale request distribution across thousands of IPs, ensuring low footprint detection.

Performance Monitoring & Continuous Improvement

• Develop real-time monitoring dashboards for scraping performance, detection rates, and error tracking.
• Implement test-driven development (TDD) and automated recovery mechanisms to prevent scraper downtime.
• Improve failover mechanisms to avoid disruptions due to rate limits, bans, or CAPTCHA triggers.
• Research new AI/ML-based anti-bot evasion techniques to stay ahead of evolving detection systems.

Requirements & Qualifications

Technical Expertise

• Strong experience in web scraping & anti-bot evasion (Playwright, Puppeteer, Selenium, Scrapy, BeautifulSoup).
• Experience with asynchronous programming & scalable architectures (Python asyncio, Twisted, Golang, Node.js).
• Proficiency in Linux-based cloud systems, Kubernetes, Docker, Terraform for large-scale deployment.
• Hands-on with proxy management (residential, datacenter, P2P networks), CAPTCHA-solving techniques.
• Experience with industrial automation & data acquisition (useful for interfacing with utility APIs).

Soft Skills & Mindset

• Problem-solving mindset — you enjoy breaking and bypassing defenses in a systematic way.
• Strong background in R&D, working with open-source projects & test-driven development.
• Scalability-driven engineering approach — building for long-term efficiency, resilience, and stealth.
• Comfort with high-stakes, adversarial environments, and staying ahead of anti-bot countermeasures.

Why Join Deck?

• Competitive pay for the right skills.
• Proven leadership with a track record of big results.
• Significant ownership and autonomy in how you operate.
• A team of exceptional peers tackling complex, high-leverage problems.
• Momentum: recent fundraise by top-tier investors, massive whitespace, and accelerating traction.

Deck isn’t just a product—it’s a mission. We believe that businesses should spend more time thinking and building, and less time wrestling with data plumbing. If that vision excites you, and you’re energized by early-stage speed, ambiguity, and intensity—let’s talk.

Before applying, take a look at our Constitution. If you don't dislike it, there's a good chance you’ll love working here.

J-18808-Ljbffr