Docs › Guides › Handling Anti-Bot Protection

Handling Anti-Bot Protection

Modern websites use advanced bot detection. This guide explains how FourA handles anti-bot systems and how to maximize your success rate.

How Bot Detection Works

Websites use several layers of protection:

  • IP reputation: Data centers and known proxy IPs get blocked
  • TLS fingerprinting: Each HTTP client has a unique TLS signature
  • Browser fingerprinting: JavaScript checks for headless browser indicators
  • Behavioral analysis: Request patterns, timing, and navigation flow
  • CAPTCHAs: Visual challenges as a last line of defense

Common anti-bot providers include Cloudflare, DataDome, PerimeterX, and Akamai Bot Manager.

How FourA Helps

TLS Fingerprint Mimicry

The single task type uses curl-impersonate, which replicates the exact TLS fingerprint of real browsers. This passes the most common TLS-based checks without the overhead of running a full browser.

Real Browser Rendering

The browser task type runs actual headless Chrome. Combined with anti-detection patches, it passes most JavaScript-based fingerprint checks.

Proxy Rotation

The proxy task type automatically rotates through residential and data center proxies. If one IP gets blocked, the next attempt uses a different one.

Strategy by Protection Level

Low Protection (most sites)

Use single type. The TLS fingerprint mimicry is sufficient.

{"url": "https://news-site.com/article", "type": "single"}

Medium Protection (Cloudflare, basic WAF)

Use browser type to pass JavaScript challenges:

{"url": "https://protected-site.com/data", "type": "browser"}

High Protection (DataDome, PerimeterX)

Use proxy type with residential proxies:

{
  "url": "https://heavily-protected.com/prices",
  "type": "proxy",
  "options": {"proxyCountry": "US"}
}

Best Practices

  1. Start simple, escalate gradually. Try single first. Only switch to browser or proxy when needed.

  2. Respect rate limits. Even with proxy rotation, sending hundreds of requests per second to a single site will trigger behavioral detection. Space your requests by at least 1-2 seconds.

  3. Rotate user agents. For single tasks, use the headers option to vary the User-Agent string.

  4. Use geo-targeted proxies. Some sites serve different content by region or block traffic from certain countries. Set proxyCountry to match the target audience.

  5. Monitor success rates. Check the Dashboard analytics to track your success rate over time. A sudden drop usually means the target site updated its protection.

What FourA Can't Bypass

Some scenarios require additional handling outside the API:

  • Login-protected content: FourA doesn't manage sessions or cookies across requests (coming soon)
  • Interactive CAPTCHAs: reCAPTCHA v3 and hCaptcha require solving services
  • Sites with legal restrictions: Always ensure your data collection complies with the target site's terms of service and applicable laws

Next Steps

Last updated: April 8, 2026
← Choosing the Right Task Type