# Protected sites

> URL: https://foura.ai/docs/guides/protected-sites
> Updated: 2026-11-11
> Index: https://foura.ai/llms.txt

Modern websites use advanced bot detection. This guide explains how FourA gets you the page on sites that check who is asking, and how to raise your success rate.

## How Bot Detection Works

Websites use several layers of protection:

- **IP reputation**: Data centers and known proxy addresses get blocked
- **Wire fingerprinting**: Each HTTP client has a unique handshake signature that sites can detect
- **Browser fingerprinting**: JavaScript checks for headless browser indicators
- **Behavioral analysis**: Request patterns, timing, and navigation flow
- **Verification pages**: a visual task the visitor has to complete

The response names the system that ran a check; [Site checks](https://foura.ai/docs/api/site-checks) lists them.

## Fastest Path: Auto

If you don't know the protection level yet, call `/api/auto/` with a `validate.data.accept` substring that only the real page carries. Auto walks a cost-aware ladder (a rotated proxy, then a browser through a proxy; with `forceProxy: false`, a cheap direct probe and a direct browser render come first) and stops at the first rung that returns a response your rules accept. On repeat calls to the same host, a warm session is replayed instead, so the second hit is cheap.

```bash
curl -X POST https://eu.api.foura.ai/api/auto/ \
  -H "X-API-Key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://protected-site.com/product/42",
    "validate": {"data": {"accept": ["Add to cart"]}}
  }'
```

Auto recognizes the common challenge pages and keeps climbing when it meets one. Your `validate.data.accept` string catches the rest: a check page it doesn't know, a login wall, or a page without the content you need. See the [Smart Fetch guide](https://foura.ai/docs/guides/smart-fetch) for the full walkthrough.

## How FourA Helps at Each Layer

### Realistic Wire-Level Requests

The single endpoint (`POST /api/single/`) emits handshake characteristics that match a real browser. Sites answer it the way they answer a browser, without the overhead of running one.

Enable `unblocker` to also inject realistic browser headers (User-Agent, Sec-Ch-Ua, Sec-Fetch-*, Accept-Encoding). `unblocker` is on by default; set `false` only to send your own headers with none added.

```json
{
  "method": "GET",
  "url": "https://protected-site.com/data",
  "unblocker": true
}
```

### Real Browser Rendering

The browser endpoint (`POST /api/browser/`) runs a full Chrome browser instance. It runs the page's JavaScript the way a visitor's browser does. `unblocker` on Browser completes the checks a page asks for before it loads (Turnstile and similar gates); leave it on unless you want the challenge page back as it came.

### Proxy Rotation

The proxy endpoint (`POST /api/proxy/`) rotates through FourA's standard pool of exits. When a site refuses an exit or the exit doesn't answer, the next attempt goes out through a different one. Use `ignoreProxies` on a follow-up call to skip exits you already burned; use `maxTries` (default 5, max 90) to control how hard it works.

Premium exits are used only when a request asks for them. `exitClass: "premium"` lets Proxy move on to a premium exit once the standard pool can't reach the page, on a plan that includes premium exits. A successful response's `exitClass` says which kind of exit delivered, and once your premium traffic for the period is spent, the request goes on in the standard pool and says `standard`. See [exitClass](https://foura.ai/docs/api/endpoints#exitclass).

### Country-Scoped Exits

Pass `exitCountries` on `/api/proxy/` to restrict selection to proxies whose target-visible country matches a strict allowlist. Values are two-letter codes (`["CZ", "GB"]`), trimmed, uppercased, and deduplicated. The standard pool never falls back to an unrequested country; if the current pool has no match, the response returns `code: "no_eligible_proxy"` with the normalized scope in `details.exitCountries` so you can retry later without loosening the requirement. A premium exit stays inside the list too: with `exitClass: premium`, a premium exit leaves from one of your countries, and `exitCountry` in the response names it. Country scoping is included from the Startup plan up. On a plan without it, a call that sends `exitCountries` is refused with a `403` and `X-FourA-Limit: plan_limit_feature`.

```bash
curl -X POST https://eu.api.foura.ai/api/proxy/ \
  -H "X-API-Key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "maxTries": 5,
    "exitCountries": ["CZ", "GB"],
    "request": {"method": "GET", "url": "https://target.example/pricing"}
  }'
```

The response includes `exitCountry` when scoping was requested. Verify it belongs to your allowlist before trusting the payload, then reuse the returned `proxy` ID on any follow-up Browser call so the JavaScript render happens through the same exit.

## FourA Tells You What Stopped You

You don't have to guess which system blocked a request. When a target runs a bot check, the response names it.

- `POST /api/single/` and `POST /api/proxy/` return a `defense` object: `defense.vendor` is the system, `defense.solved` says whether the check was cleared, and `defense.present` lists everything recognised on that response.
- `POST /api/browser/` returns `defenseSolved` plus `defenses.present` and `defenses.cleared`.

```json
{
  "status": 200,
  "data": "<!doctype html>...",
  "defense": {
    "vendor": "sgcaptcha",
    "solved": true,
    "present": ["sgcaptcha"],
    "cookie": "_I_=<clearance>"
  }
}
```

Two rules follow from it:

1. **`solved: false` means the body may be the challenge, not the page.** FourA never presents a challenge page as content, so check the flag before parsing.
2. **A solve hands you the clearance.** When `defense.cookie` is present, send it back as a `Cookie` header on the same exit with the same User-Agent and the follow-up requests skip the check entirely.

FourA recognises the common check systems, including the first-party checks of eBay, Reddit and Amazon. Recognising is wider than clearing: a system we can name but not clear is reported and never raises what the request costs. When the page that came back is that system's check page, even at HTTP 200, the request isn't billed and the `X-FourA-Check-Page` header names it. See [Site checks](https://foura.ai/docs/api/site-checks) for every field, the current clear-versus-detect split, and a replay example.

Amazon's one-button "continue shopping" page is answered for you rather than cleared. Single, Proxy and Browser press the button on the same exit and return the real page. With `unblocker` on (the default), Single and Proxy report it in `defense` as `retry: "amazon-continue"` with `solved: false`, and the price doesn't change. Reddit's light JavaScript check works the same way on Single and Proxy: they answer it on the same exit, return the real page, and report `retry: "reddit-js-check"` with `solved: false` at the same price.

## Strategy by Protection Level

### Unknown Protection

Use auto. It starts on a rotated proxy and only escalates as far as the target forces it, so you pay for the discovery once per host.

### Low Protection (most sites)

Use the single endpoint with `unblocker`. The wire-level match is enough.

```bash
curl -X POST https://eu.api.foura.ai/api/single/ \
  -H "X-API-Key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"method": "GET", "url": "https://news-site.com/article", "unblocker": true}'
```

### Medium: a challenge page or a basic firewall

Use the browser endpoint to pass JavaScript challenges:

```bash
curl -X POST https://eu.api.foura.ai/api/browser/ \
  -H "X-API-Key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://protected-site.com/data", "timeout_ms": 15000}'
```

### High: behaviour and fingerprint checks

Use the proxy endpoint with multiple retry attempts:

```bash
curl -X POST https://eu.api.foura.ai/api/proxy/ \
  -H "X-API-Key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "maxTries": 10,
    "request": {
      "method": "GET",
      "url": "https://heavily-protected.com/prices",
      "unblocker": true
    }
  }'
```

For chained challenge pages ("Just a moment", a security checkpoint) where you need the rendered page after the challenge clears, see the [MCP Recipes](https://foura.ai/docs/mcp/recipes). The "Protected page: proxy first, browser when JavaScript is needed" recipe shows the exact proxy-then-browser handoff.

## Best Practices

1. **Start with auto for unknown targets.** Pass a `validate` rule, let the ladder pick the cheapest rung, then read `meta.rung` in the response to see which engine worked. Once you know, call that engine directly for repeat traffic.

2. **Reuse the winning session.** After an auto call, the returned `session` (proxy ID + cookies + userAgent) can be replayed through `/api/single/` or `/api/browser/` for follow-up pages on the same host, at that endpoint's own price: 2 credits on Single with `unblocker`, 5 on Browser (10 when the call completes a verification step the page asks for).

3. **Respect rate limits.** Even with proxy rotation, sending hundreds of requests per second to a single site will trigger behavioral detection. Space your requests by at least 1 to 2 seconds.

4. **Keep `unblocker` on.** On Single and Proxy, `unblocker: true` (the default) adds realistic browser headers. On Browser it turns on the challenge solver. Turn it off only when you specifically need a request with no added headers or a raw challenge page.

5. **Monitor success rates.** Check the [Dashboard](https://foura.ai/dashboard) metrics to track your success rate over time. A sudden drop usually means the target site updated its protection.

6. **Skip burned exits.** If a `/api/proxy/` or `/api/auto/` call returned a proxy ID that then started failing, pass it in `ignoreProxies` on the next call so FourA picks a different exit.

7. **Read `defense` before you retry.** The vendor name tells you whether a different browser profile is worth trying, whether you need a full render, or whether the check is one nobody clears without a solving service.

8. **Change the browser you present.** Some targets accept one browser and refuse another. Set `browser`, `os`, or `version` on Single and Proxy, and read `GET /api/profiles` for the current catalogue. Details in the [endpoint reference](https://foura.ai/docs/api/endpoints).

## Limits

Some scenarios require additional handling outside the API:

- **Login-protected content**: FourA doesn't manage long-lived logins for you. The browser endpoint accepts `cookies` per-request; carry your session cookies in yourself.
- **Interactive verification tasks**: FourA recognises the visual ones and reports them in `defense.present`, but doesn't complete them. Turnstile is handled by Browser.
- **Content limited to certain countries**: use `exitCountries` on `/api/proxy/` to pin selection to allowed countries. Sites that also check which network an address belongs to (some country-licensed bookmakers, certain government services) may still refuse exits from the right country. With `exitClass: premium` and one country in `exitCountries`, a premium exit can also leave from a region, city or network you name: see [Exit location](https://foura.ai/docs/api/endpoints#exit-location). When the pool has no exit in the countries you allowed, the request returns `no_eligible_proxy`.
- **Sites with legal restrictions**: Always ensure your data collection complies with the target site's terms of service and applicable laws.

## Next Steps

- [Smart Fetch (Auto)](https://foura.ai/docs/guides/smart-fetch): Deep dive on `/api/auto/`
- [Choosing the Right Endpoint](https://foura.ai/docs/guides/choosing-task-type): Decision guide for endpoints
- [Common Issues](https://foura.ai/docs/troubleshooting/common-issues): Fix 403s and verification pages
- [API Endpoints](https://foura.ai/docs/api/endpoints): Full parameter reference including `exitCountries`
- [Site checks](https://foura.ai/docs/api/site-checks): The `defense` field, the list of systems, and clearance replay
- [MCP Recipes](https://foura.ai/docs/mcp/recipes): Workflow templates including the proxy-first-then-browser chain
