How to Run Browser Automation Without Getting Blocked
Successful browser automation without getting blocked requires defeating five detection layers simultaneously: rate limiting, IP reputation checks, browser fingerprinting, behavioral analysis, and CAPTCHAs. No single technique is enough — sites like Amazon, Google, and LinkedIn stack these defenses. Below are eight proven approaches with working code examples in Node.js and Puppeteer, ordered from quick fixes to comprehensive solutions that combine antidetect browser profiles with automation frameworks.
Why Browser Automation Gets Blocked
Before diving into solutions, understanding what you’re up against makes every fix more effective. Modern anti-bot systems operate on multiple detection vectors simultaneously:
| Detection Layer | What It Checks | Blocking Speed | Difficulty to Bypass |
|---|---|---|---|
| Rate Limiting | Requests per second/minute from one source | Instant | 🟢 Easy |
| IP Reputation | Datacenter IPs, known proxy ranges, abuse history | Instant | 🟡 Medium |
| Browser Fingerprinting | Canvas, WebGL, fonts, navigator properties, plugins | 1-5 requests | 🔴 Hard |
| Behavioral Analysis | Mouse movement, scroll patterns, click timing, navigation flow | 5-20 requests | 🔴 Hard |
| CAPTCHA Challenges | Human verification when suspicion threshold is crossed | Triggered | 🟡 Medium (with services) |
Most blocked automation scripts fail because they address only one or two of these layers. A script with perfect fingerprinting still gets blocked if it sends 100 requests per second from a datacenter IP. A well-throttled script with residential proxies still fails if the site detects Puppeteer’s Selenium browser fingerprint leaks. You need a layered defense to match the layered detection.
1. Request Throttling and Human-Like Pacing
Rate limiting is the simplest detection vector and the easiest to defeat. The key is introducing randomized delays that mimic human browsing patterns — not constant intervals, which are equally suspicious.
// Human-like delay utility with randomized timing
function humanDelay(minMs = 1500, maxMs = 4500) {
const delay = Math.floor(Math.random() * (maxMs - minMs + 1)) + minMs;
return new Promise(resolve => setTimeout(resolve, delay));
}
// Gaussian-distributed delay for more natural timing
function gaussianDelay(meanMs = 3000, stdDevMs = 1000) {
// Box-Muller transform for normal distribution
const u1 = Math.random();
const u2 = Math.random();
const z = Math.sqrt(-2 * Math.log(u1)) * Math.cos(2 * Math.PI * u2);
const delay = Math.max(500, Math.round(meanMs + z * stdDevMs));
return new Promise(resolve => setTimeout(resolve, delay));
}
// Usage in a scraping loop
async function scrapeWithThrottling(page, urls) {
const results = [];
for (const url of urls) {
await page.goto(url, { waitUntil: 'networkidle2' });
const data = await page.evaluate(() => {
return {
title: document.title,
content: document.querySelector('main')?.innerText || ''
};
});
results.push(data);
// Random delay between page loads
await gaussianDelay(3000, 1200);
}
return results;
}The gaussian distribution is important because real humans don’t wait exactly 3 seconds between actions — sometimes they read quickly (1.5 seconds), sometimes they pause to think (6+ seconds). Flat random distributions are better than fixed delays, but gaussian more closely models actual behavior.
2. Proxy Rotation with Session Stickiness
IP-based blocking is the second most common defense. The solution is rotating residential proxies — but naive rotation causes its own problems. Switching IPs mid-session triggers anomaly detection, since real users don’t change geographic locations between page loads.
const puppeteer = require('puppeteer');
// Proxy configuration with sticky sessions
const PROXY_CONFIG = {
host: 'gate.smartproxy.com',
port: 7000,
username: 'user',
password: 'pass'
};
async function createSessionBrowser(sessionId) {
// Sticky session: same IP for the entire browsing session
const proxyAuth = `${PROXY_CONFIG.username}-session-${sessionId}:${PROXY_CONFIG.password}`;
const browser = await puppeteer.launch({
headless: 'new',
args: [
`--proxy-server=${PROXY_CONFIG.host}:${PROXY_CONFIG.port}`,
'--disable-blink-features=AutomationControlled',
'--no-first-run',
'--no-default-browser-check'
]
});
const page = await browser.newPage();
await page.authenticate({
username: proxyAuth.split(':')[0],
password: PROXY_CONFIG.password
});
return { browser, page };
}
// Rotate sessions across tasks, not within them
async function scrapeWithRotation(urls, batchSize = 5) {
const results = [];
for (let i = 0; i < urls.length; i += batchSize) {
const batch = urls.slice(i, i + batchSize);
const sessionId = `session_${Date.now()}_${i}`;
const { browser, page } = await createSessionBrowser(sessionId);
for (const url of batch) {
try {
await page.goto(url, { waitUntil: 'domcontentloaded', timeout: 30000 });
const data = await page.evaluate(() => document.title);
results.push({ url, title: data });
await humanDelay(2000, 5000);
} catch (err) {
console.error(`Failed: ${url} - ${err.message}`);
}
}
await browser.close();
// Longer delay between session rotations
await humanDelay(5000, 10000);
}
return results;
}The critical insight is session stickiness: keep the same IP for a batch of related requests (like browsing a single site), then rotate when starting a new session. This matches how real users behave — they don’t teleport between countries mid-browsing.
3. Fingerprint Spoofing and Headless Detection Evasion
This is where most automation scripts fail. Default Puppeteer and Selenium installations leak dozens of signals that identify them as automated: the navigator.webdriver flag, missing plugin arrays, Chrome-specific object inconsistencies, and distinctive WebGL fingerprinting patterns.
const puppeteer = require('puppeteer');
async function createStealthBrowser() {
const browser = await puppeteer.launch({
headless: 'new',
args: [
'--disable-blink-features=AutomationControlled',
'--disable-features=IsolateOrigins,site-per-process',
'--window-size=1920,1080',
'--no-first-run',
'--no-default-browser-check',
'--disable-infobars'
]
});
const page = await browser.newPage();
// Set realistic viewport
await page.setViewport({ width: 1920, height: 1080, deviceScaleFactor: 1 });
// Override navigator.webdriver
await page.evaluateOnNewDocument(() => {
Object.defineProperty(navigator, 'webdriver', {
get: () => undefined
});
// Fix chrome.runtime to look like a real browser
window.chrome = {
runtime: {
onMessage: { addListener: () => {} },
sendMessage: () => {}
}
};
// Override plugins to match typical Chrome
Object.defineProperty(navigator, 'plugins', {
get: () => [
{ name: 'Chrome PDF Plugin', filename: 'internal-pdf-viewer' },
{ name: 'Chrome PDF Viewer', filename: 'mhjfbmdgcfjbbpaeojofohoefgiehjai' },
{ name: 'Native Client', filename: 'internal-nacl-plugin' }
]
});
// Override languages
Object.defineProperty(navigator, 'languages', {
get: () => ['en-US', 'en']
});
// Fix permissions query
const originalQuery = window.navigator.permissions.query;
window.navigator.permissions.query = (parameters) => {
if (parameters.name === 'notifications') {
return Promise.resolve({ state: Notification.permission });
}
return originalQuery(parameters);
};
});
// Set realistic user agent
await page.setUserAgent(
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 ' +
'(KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36'
);
return { browser, page };
}This handles the most common detection vectors, but sophisticated anti-bot systems like DataDome and PerimeterX check dozens more signals. The puppeteer-extra-plugin-stealth package patches about 10 known leak points, which helps — but it’s an arms race. Every time stealth plugins patch a leak, anti-bot vendors add new detection vectors.
4. Human-Like Mouse Movement and Keyboard Input
Behavioral analysis examines how you interact with a page, not just what you request. Bot traffic shows telltale patterns: instant clicks without mouse movement, perfectly uniform typing speed, and teleporting cursor positions.
// Bezier curve mouse movement for natural paths
async function humanMouseMove(page, targetX, targetY) {
const mouse = page.mouse;
const steps = Math.floor(Math.random() * 15) + 10;
// Get current position (default to random start)
const startX = Math.floor(Math.random() * 500) + 100;
const startY = Math.floor(Math.random() * 400) + 100;
// Control point for bezier curve (adds natural arc)
const cpX = (startX + targetX) / 2 + (Math.random() - 0.5) * 200;
const cpY = (startY + targetY) / 2 + (Math.random() - 0.5) * 200;
for (let i = 0; i <= steps; i++) {
const t = i / steps;
const x = Math.round(
(1 - t) * (1 - t) * startX + 2 * (1 - t) * t * cpX + t * t * targetX
);
const y = Math.round(
(1 - t) * (1 - t) * startY + 2 * (1 - t) * t * cpY + t * t * targetY
);
await mouse.move(x, y);
// Vary speed — slower near the target (Fitts's law)
const delay = Math.floor(Math.random() * 20) + 5 + (t > 0.8 ? 15 : 0);
await new Promise(r => setTimeout(r, delay));
}
}
// Natural typing with variable speed and occasional corrections
async function humanType(page, selector, text) {
await page.click(selector);
await new Promise(r => setTimeout(r, 300 + Math.random() * 200));
for (let i = 0; i < text.length; i++) {
// Occasional brief pauses (thinking)
if (Math.random() < 0.05) {
await new Promise(r => setTimeout(r, 500 + Math.random() * 1000));
}
await page.keyboard.type(text[i], {
delay: 50 + Math.random() * 120 // Variable keystroke timing
});
}
}
// Natural scrolling behavior
async function humanScroll(page) {
const scrolls = Math.floor(Math.random() * 4) + 2;
for (let i = 0; i < scrolls; i++) {
const distance = Math.floor(Math.random() * 400) + 200;
await page.evaluate((d) => {
window.scrollBy({ top: d, behavior: 'smooth' });
}, distance);
await humanDelay(800, 2500);
}
}The bezier curve movement is particularly important. Real mouse paths are curved, not straight lines. They also decelerate near the target (Fitts’s law) — a pattern that bot detection systems specifically look for.
5. CAPTCHA Handling Strategies
CAPTCHAs are the last line of defense. When a site serves one during automation, you have three realistic options:
Option A: CAPTCHA Solving Services
const axios = require('axios');
async function solveCaptcha(siteKey, pageUrl) {
// Submit CAPTCHA task to solving service
const createTask = await axios.post('https://api.2captcha.com/createTask', {
clientKey: process.env.CAPTCHA_API_KEY,
task: {
type: 'RecaptchaV2TaskProxyless',
websiteURL: pageUrl,
websiteKey: siteKey
}
});
const taskId = createTask.data.taskId;
// Poll for solution (typically 15-45 seconds)
let solution = null;
for (let attempt = 0; attempt < 30; attempt++) {
await new Promise(r => setTimeout(r, 5000));
const result = await axios.post('https://api.2captcha.com/getTaskResult', {
clientKey: process.env.CAPTCHA_API_KEY,
taskId: taskId
});
if (result.data.status === 'ready') {
solution = result.data.solution.gRecaptchaResponse;
break;
}
}
return solution;
}
// Inject solution into page
async function handleCaptcha(page) {
const siteKey = await page.evaluate(() => {
const el = document.querySelector('.g-recaptcha');
return el ? el.getAttribute('data-sitekey') : null;
});
if (!siteKey) return false;
const token = await solveCaptcha(siteKey, page.url());
if (!token) return false;
await page.evaluate((t) => {
document.getElementById('g-recaptcha-response').value = t;
// Trigger the callback if it exists
if (typeof window.captchaCallback === 'function') {
window.captchaCallback(t);
}
}, token);
return true;
}Option B: Reduce CAPTCHA Triggers
Better than solving CAPTCHAs is avoiding them entirely. Sites serve CAPTCHAs based on a risk score — keep your score low with clean IPs, realistic fingerprints, and natural behavior patterns. Combining methods 1 through 4 above often eliminates CAPTCHAs for sites that use risk-based triggering rather than blanket challenges.
6. Request Header and TLS Fingerprint Management
Advanced anti-bot systems inspect HTTP headers and TLS handshake characteristics. Default Puppeteer headers differ from real Chrome, and the TLS fingerprint (JA3/JA4 hash) of Node.js differs from a real browser.
async function setRealisticHeaders(page) {
// Match real Chrome header order and values
await page.setExtraHTTPHeaders({
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8',
'Accept-Encoding': 'gzip, deflate, br, zstd',
'Accept-Language': 'en-US,en;q=0.9',
'Cache-Control': 'max-age=0',
'Sec-Ch-Ua': '"Chromium";v="127", "Not)A;Brand";v="99", "Google Chrome";v="127"',
'Sec-Ch-Ua-Mobile': '?0',
'Sec-Ch-Ua-Platform': '"Windows"',
'Sec-Fetch-Dest': 'document',
'Sec-Fetch-Mode': 'navigate',
'Sec-Fetch-Site': 'none',
'Sec-Fetch-User': '?1',
'Upgrade-Insecure-Requests': '1'
});
// Override referer naturally during navigation
page.on('request', request => {
if (request.isNavigationRequest() && request.redirectChain().length === 0) {
// First navigation — no referer (direct visit)
}
// Subsequent navigations will naturally inherit referer from the page
});
}Header order matters more than most developers realize. Anti-bot systems like Cloudflare check whether the Accept header appears before Accept-Language, whether Sec-Ch-Ua headers are present (they should be for Chrome 89+), and whether the overall header set matches what the claimed user agent would actually send.
7. Session and Cookie Management
Maintaining persistent sessions across automation runs helps build trust scores with sites that track returning visitors. A browser that always starts with zero cookies and no history looks suspicious. Proper session isolation means each automated profile maintains its own persistent cookie jar and browsing state.
const fs = require('fs');
const path = require('path');
class SessionManager {
constructor(profileDir) {
this.profileDir = profileDir;
this.cookieFile = path.join(profileDir, 'cookies.json');
this.storageFile = path.join(profileDir, 'localStorage.json');
}
async loadSession(page) {
// Restore cookies from previous session
if (fs.existsSync(this.cookieFile)) {
const cookies = JSON.parse(fs.readFileSync(this.cookieFile, 'utf8'));
await page.setCookie(...cookies);
console.log(`Loaded ${cookies.length} cookies from previous session`);
}
// Restore localStorage
if (fs.existsSync(this.storageFile)) {
const storage = JSON.parse(fs.readFileSync(this.storageFile, 'utf8'));
await page.evaluateOnNewDocument((data) => {
for (const [key, value] of Object.entries(data)) {
localStorage.setItem(key, value);
}
}, storage);
}
}
async saveSession(page) {
// Save current cookies
const cookies = await page.cookies();
fs.writeFileSync(this.cookieFile, JSON.stringify(cookies, null, 2));
// Save localStorage
const storage = await page.evaluate(() => {
const data = {};
for (let i = 0; i < localStorage.length; i++) {
const key = localStorage.key(i);
data[key] = localStorage.getItem(key);
}
return data;
});
fs.writeFileSync(this.storageFile, JSON.stringify(storage, null, 2));
}
}
// Usage
async function automateWithSession(profileId) {
const session = new SessionManager(`./profiles/${profileId}`);
const { browser, page } = await createStealthBrowser();
await session.loadSession(page);
// ... perform automation tasks ...
await session.saveSession(page);
await browser.close();
}This approach works for simple cases, but managing dozens or hundreds of profiles — each with unique fingerprints, cookies, proxies, and browsing history — quickly becomes unmanageable with hand-rolled code.
8. Connecting Automation to Antidetect Browser Profiles
The most robust approach to browser automation without getting blocked combines all the previous techniques into a pre-configured antidetect browser profile, then connects your automation framework to that profile rather than launching a raw Puppeteer instance. This eliminates the need to manually spoof dozens of fingerprint parameters — the antidetect browser handles it at the engine level.
With Sendwin Browser, each profile runs as an isolated Chromium instance with a consistent, realistic fingerprint. The Automation API exposes a standard Puppeteer/Selenium/Playwright connection endpoint on each running profile, so your scripts connect to an already-configured browser rather than trying to patch a stock one:
const puppeteer = require('puppeteer');
async function connectToAntidetectProfile(profilePort) {
// Connect to a running antidetect browser profile's automation endpoint
// The profile already has: unique fingerprint, assigned proxy,
// persistent cookies, isolated storage
const browser = await puppeteer.connect({
browserWSEndpoint: `ws://127.0.0.1:${profilePort}/devtools/browser`,
defaultViewport: null // Use profile's configured viewport
});
const pages = await browser.pages();
const page = pages[0] || await browser.newPage();
return { browser, page };
}
// Full automation flow with antidetect profile
async function scrapeWithProfile(profilePort, urls) {
const { browser, page } = await connectToAntidetectProfile(profilePort);
const results = [];
for (const url of urls) {
try {
await page.goto(url, { waitUntil: 'networkidle2', timeout: 30000 });
// Natural interaction — the fingerprint is already handled
await humanScroll(page);
await gaussianDelay(2000, 800);
const data = await page.evaluate(() => ({
title: document.title,
content: document.querySelector('article')?.innerText || '',
links: Array.from(document.querySelectorAll('a[href]')).map(a => a.href)
}));
results.push({ url, ...data });
await gaussianDelay(3000, 1500);
} catch (err) {
console.error(`Error on ${url}: ${err.message}`);
results.push({ url, error: err.message });
}
}
// Don't close — the profile persists for the next run
browser.disconnect();
return results;
}The advantage over DIY fingerprint spoofing is consistency and depth. Methods 3 through 6 patch individual detection vectors at the JavaScript level, but antidetect browsers modify the fingerprint at the browser engine level — affecting canvas rendering, WebGL output, audio context, font enumeration, and dozens of other low-level signals that JavaScript-level patches can’t reach. Sophisticated anti-bot systems from Cloudflare and DataDome specifically test for inconsistencies between JavaScript-reported values and actual browser behavior, which is exactly the gap that engine-level fingerprinting closes.
Comparison: DIY Stealth vs. Antidetect Browser Automation
| Factor | DIY Puppeteer Stealth | Antidetect Browser + Automation API |
|---|---|---|
| Setup time | Hours of configuration per script | Minutes per profile |
| Fingerprint depth | JavaScript-level only (~15 parameters) | Engine-level (50+ parameters) |
| Canvas/WebGL consistency | Spoofed values may conflict with GPU info | Consistent across all APIs |
| Cookie persistence | Manual file management | Automatic per profile |
| Proxy assignment | Global or per-launch config | Per-profile, persistent |
| Scaling to 100+ profiles | Complex infrastructure required | Built-in profile management |
| Maintenance | Constant updates as detection evolves | Maintained by browser vendor |
| Cost | Free (code) + proxies | From $9.99/mo ($6.99/mo annual) + proxies |
Putting It All Together: Complete Automation Script
Here’s a complete script combining throttling, session management, human-like behavior, and error handling:
const puppeteer = require('puppeteer');
const fs = require('fs');
// ---- Configuration ----
const CONFIG = {
concurrency: 2, // Parallel sessions
batchSize: 5, // URLs per session before IP rotation
minDelay: 2000, // Minimum delay between requests (ms)
maxDelay: 6000, // Maximum delay between requests (ms)
sessionPauseMin: 8000, // Minimum pause between sessions
sessionPauseMax: 15000, // Maximum pause between sessions
maxRetries: 3, // Retries per URL on failure
timeout: 30000 // Page load timeout
};
// ---- Utility Functions ----
function humanDelay(min = CONFIG.minDelay, max = CONFIG.maxDelay) {
const delay = Math.floor(Math.random() * (max - min + 1)) + min;
return new Promise(r => setTimeout(r, delay));
}
async function humanScroll(page) {
const scrolls = Math.floor(Math.random() * 3) + 1;
for (let i = 0; i < scrolls; i++) {
await page.evaluate((d) => {
window.scrollBy({ top: d, behavior: 'smooth' });
}, Math.floor(Math.random() * 500) + 200);
await humanDelay(600, 1800);
}
}
// ---- Main Scraping Engine ----
async function scrapeUrls(urls, outputFile) {
const results = [];
const failed = [];
// Process in batches
for (let i = 0; i < urls.length; i += CONFIG.batchSize) {
const batch = urls.slice(i, i + CONFIG.batchSize);
console.log(`\n--- Session ${Math.floor(i / CONFIG.batchSize) + 1} ---`);
console.log(`Processing ${batch.length} URLs (${i + 1}-${i + batch.length} of ${urls.length})`);
// Connect to antidetect profile or launch stealth browser
let browser, page;
try {
// Option A: connect to an antidetect profile's automation endpoint
// browser = await puppeteer.connect({
// browserWSEndpoint: 'ws://127.0.0.1:PORT/devtools/browser'
// });
// Option B: launch stealth browser
browser = await puppeteer.launch({
headless: 'new',
args: [
'--disable-blink-features=AutomationControlled',
'--window-size=1920,1080',
'--no-first-run'
]
});
page = (await browser.pages())[0] || await browser.newPage();
await page.setViewport({ width: 1920, height: 1080 });
await page.evaluateOnNewDocument(() => {
Object.defineProperty(navigator, 'webdriver', { get: () => undefined });
});
} catch (err) {
console.error('Browser launch failed:', err.message);
continue;
}
// Process each URL in the batch
for (const url of batch) {
let success = false;
for (let attempt = 1; attempt <= CONFIG.maxRetries; attempt++) {
try {
console.log(` Fetching: ${url} (attempt ${attempt})`);
await page.goto(url, {
waitUntil: 'domcontentloaded',
timeout: CONFIG.timeout
});
// Human-like page interaction
await humanScroll(page);
// Extract data
const data = await page.evaluate(() => ({
title: document.title,
url: window.location.href,
headings: Array.from(document.querySelectorAll('h1, h2'))
.map(h => ({ tag: h.tagName, text: h.textContent.trim() })),
wordCount: document.body?.innerText.split(/\s+/).length || 0
}));
results.push(data);
success = true;
console.log(` ✅ Success: ${data.title}`);
break;
} catch (err) {
console.error(` ❌ Attempt ${attempt} failed: ${err.message}`);
if (attempt < CONFIG.maxRetries) {
await humanDelay(5000, 10000); // Longer delay before retry
}
}
}
if (!success) failed.push(url);
await humanDelay(); // Delay between URLs
}
await browser.close();
// Session rotation pause
if (i + CONFIG.batchSize < urls.length) {
console.log('Session complete. Rotating...');
await humanDelay(CONFIG.sessionPauseMin, CONFIG.sessionPauseMax);
}
}
// Save results
fs.writeFileSync(outputFile, JSON.stringify({ results, failed }, null, 2));
console.log(`\nDone: ${results.length} succeeded, ${failed.length} failed`);
console.log(`Results saved to ${outputFile}`);
}
// ---- Entry Point ----
const urls = [
'https://example.com/page-1',
'https://example.com/page-2',
'https://example.com/page-3'
];
scrapeUrls(urls, 'results.json').catch(console.error);🏆 Send.win Verdict
DIY stealth scripting works for simple automation tasks, but scaling to dozens of profiles while maintaining undetectable fingerprints across every detection vector is a full-time maintenance burden. Sendwin Browser handles fingerprint generation, session isolation, and proxy assignment at the engine level — then exposes a standard Puppeteer/Selenium/Playwright endpoint so your existing automation code connects to a pre-configured profile instead of fighting detection patchwork. The Automation API is available on both the Pro plan ($9.99/month, or $6.99/month annual) with 150 profiles and 5GB proxy bandwidth, and the Team plan ($29.99/month, or $20.99/month annual) with 500 profiles, 20GB bandwidth, and 16 seats.
Try Send.win free today — 30-day free trial, no credit card. Connect Puppeteer in minutes.
Frequently Asked Questions
Which automation framework is hardest to detect — Puppeteer, Selenium, or Playwright?
Playwright is generally hardest to detect out of the box because it doesn’t set the navigator.webdriver flag by default and has fewer well-known fingerprint leaks. However, all three are detectable by sophisticated anti-bot systems without additional stealth measures. The framework matters less than how you configure it — any of the three can be made stealthy with proper fingerprint spoofing, and any of the three gets caught instantly without it.
Is it legal to scrape websites with automated browsers?
Web scraping legality depends on jurisdiction, the website’s terms of service, and the type of data being collected. In the US, the LinkedIn v. hiQ Labs ruling established that scraping publicly available data is generally permissible. However, bypassing technical access controls (like CAPTCHAs) may violate the Computer Fraud and Abuse Act. Always review a site’s robots.txt and terms of service, and consult legal counsel for commercial scraping operations.
How many requests per minute can I send without getting blocked?
There’s no universal number — it depends entirely on the target site. Amazon might throttle after 15-20 requests per minute from a single IP, while smaller sites may tolerate 60+. Start with 3-5 requests per minute and gradually increase while monitoring for CAPTCHAs, 429 status codes, or soft blocks. Using residential proxies with rotation generally allows higher throughput than datacenter IPs.
Do headless browsers get blocked more than headed ones?
Yes, historically headless browsers were easier to detect due to missing GPU information, different rendering behavior, and specific JavaScript property differences. Modern Chromium’s “new headless” mode (headless: 'new' in Puppeteer) significantly reduced this gap by using the same rendering engine as headed mode. However, some anti-bot systems still detect subtle differences. Running in headed mode (or with virtual displays like Xvfb on Linux) eliminates this detection vector entirely.
Can Cloudflare detect Puppeteer even with stealth plugins?
Cloudflare’s bot detection has multiple tiers. Basic Cloudflare protection can be bypassed with stealth plugins and residential proxies. Their enterprise-tier Bot Management product (Turnstile, managed challenges) is significantly harder — it uses TLS fingerprinting, behavioral analysis, and machine learning classifiers that catch many stealth configurations. For sites behind enterprise Cloudflare, connecting to an antidetect browser profile rather than patching Puppeteer directly gives the best results because the fingerprint is consistent at the TLS and rendering engine level.
What’s the best proxy type for browser automation — datacenter, residential, or mobile?
Residential proxies offer the best balance of trust score and availability. Datacenter IPs are cheap but many are pre-flagged by anti-bot services. Mobile proxies have the highest trust scores (shared by millions of real users) but are expensive and slower. For most automation tasks, rotating residential proxies with sticky sessions provide the optimal cost-to-detection-avoidance ratio.
How do I handle sites that require JavaScript rendering for content?
Browser automation tools like Puppeteer and Playwright inherently handle JavaScript rendering since they run a full browser engine. Wait for content to load using waitUntil: 'networkidle2' or specific element selectors (page.waitForSelector('.content')) before extracting data. For single-page apps that load content dynamically on scroll, combine waitForSelector with the scroll simulation techniques shown in method 4 above.
Can I run browser automation in the cloud without a local machine?
Yes — Send.win offers cloud browser sessions that run profiles on remote infrastructure without any local installation. You can connect your Puppeteer or Playwright scripts to cloud-hosted profiles, which is particularly useful for running automation 24/7 without maintaining dedicated servers. The cloud sessions include the same fingerprint isolation as the desktop app, with bandwidth metered on your plan (5GB on Pro, 20GB on Team).
Automate Browser Automation Without Getting Blocked With Send.win
Send.win pairs isolated, fingerprint-managed browser profiles with a full Automation API, so your scripts run in profiles that look and behave like real, separate users:
- Selenium, Puppeteer & Playwright support – drive any profile programmatically (Team plan)
- Isolated profiles – each with its own fingerprint, cookies, and storage
- Built-in residential proxies – with automatic timezone, locale, and WebRTC matching
- Desktop app for Windows, macOS & Linux – plus cloud sessions when you don’t want a local install
Try the instant cloud browser demo — no install, straight from your browser. Then compare plans: a 30-day free trial with no credit card, and paid plans from $6.99/month billed annually.