Docs › Guides › Handling Anti-Bot Protection

Handling Anti-Bot Protection

Modern websites use advanced bot detection. This guide explains how FourA handles anti-bot systems and how to maximize your success rate.

How Bot Detection Works

Websites use several layers of protection:

  • IP reputation: Data centers and known proxy IPs get blocked
  • Wire fingerprinting: Each HTTP client has a unique handshake signature that sites can detect
  • Browser fingerprinting: JavaScript checks for headless browser indicators
  • Behavioral analysis: Request patterns, timing, and navigation flow
  • CAPTCHAs: Visual challenges as a last line of defense

Common anti-bot providers include Cloudflare, DataDome, PerimeterX, and Akamai Bot Manager.

How FourA Helps

Realistic Wire-Level Requests

The single endpoint (POST /api/single/) emits handshake characteristics that match a real browser. This passes the most common wire-level checks without the overhead of running a full browser.

Enable unblocker to also inject realistic browser headers (User-Agent, Sec-Ch-Ua, Sec-Fetch-*, Accept-Encoding):

{
  "method": "GET",
  "url": "https://protected-site.com/data",
  "unblocker": true
}

Real Browser Rendering

The browser endpoint (POST /api/browser/) runs a full Chrome browser instance. Combined with anti-detection patches, it passes most JavaScript-based fingerprint checks.

Proxy Rotation

The proxy endpoint (POST /api/proxy/) automatically rotates through residential and data center proxies. If one IP gets blocked, the next attempt uses a different one.

Strategy by Protection Level

Low Protection (most sites)

Use the single endpoint with unblocker. The wire-level match is enough.

curl -X POST https://eu.api.foura.ai/api/single/ \
  -H "X-API-Key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"method": "GET", "url": "https://news-site.com/article", "unblocker": true}'

Medium Protection (Cloudflare, basic WAF)

Use the browser endpoint to pass JavaScript challenges:

curl -X POST https://eu.api.foura.ai/api/browser/ \
  -H "X-API-Key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://protected-site.com/data", "timeout_ms": 15000}'

High Protection (DataDome, PerimeterX)

Use the proxy endpoint with multiple retry attempts:

curl -X POST https://eu.api.foura.ai/api/proxy/ \
  -H "X-API-Key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "maxTries": 10,
    "request": {
      "method": "GET",
      "url": "https://heavily-protected.com/prices",
      "unblocker": true
    }
  }'

For tier-1 WAF chains (Vercel Security Checkpoint, Cloudflare "Just a moment", Akamai Bot Manager) where you need the rendered page after the challenge clears, see the WAF challenge recipe for the chained proxy → browser pattern.

Best Practices

  1. Start simple, escalate gradually. Try the single endpoint first. Only switch to browser or proxy when needed.

  2. Respect rate limits. Even with proxy rotation, sending hundreds of requests per second to a single site will trigger behavioral detection. Space your requests by at least 1 to 2 seconds.

  3. Use unblocker. For single and proxy requests, enable unblocker to inject realistic browser headers automatically instead of manually setting User-Agent strings.

  4. Monitor success rates. Check the Dashboard metrics to track your success rate over time. A sudden drop usually means the target site updated its protection.

  5. Increase maxTries for tough targets. The proxy endpoint defaults to 5 attempts, but you can set up to 90 for heavily protected sites.

What FourA Can't Bypass

Some scenarios require additional handling outside the API:

  • Login-protected content: FourA doesn't manage sessions or cookies across requests (the browser endpoint supports setting cookies per-request)
  • Interactive CAPTCHAs: reCAPTCHA v3 and hCaptcha require solving services
  • Country / ASN allowlists: country-licensed sites (online bookmakers, some government services) only accept traffic from a small set of ISPs. Proxy rotation won't help; you'll need a different data source.
  • Sites with legal restrictions: Always ensure your data collection complies with the target site's terms of service and applicable laws

Next Steps

Last updated: May 20, 2026
← Choosing the Right Endpoint