← All posts
Use Case 5 min read

Aggregating Product Reviews for Sentiment at Scale

Reviews live behind JS renders, pagination, and anti-bot walls. Here's what an aggregation pipeline needs to feed a sentiment model something worth reading.

product-reviewssentiment-analysisbrowser-renderingweb-scrapingbrand-monitoring

The Challenge

Consumer brands can't rely on a single marketplace's review dashboard anymore. The same SKU carries a review score on Amazon, another on Walmart, a third on Best Buy, and yet another on Trustpilot or the brand's own Shopify store. If your product team, sentiment model, or CX ops queue only sees one of those, you're steering with a broken mirror.

That's the everyday version of the review problem: fragmentation. The reviews exist, they just live behind ten different logins, ten different rendering stacks, and ten different anti-bot postures. To feed a sentiment classifier weekly (or to trigger an alert when a bad review lands on a top listing), you need all of them, fresh, in one place.

Aggregating them is a rendering problem before it's a data problem. And most teams underestimate how big that first half is.

What Makes This Hard

A review page in 2026 looks nothing like a review page in 2018. Everything that matters gets loaded after the initial HTML:

  • The review list itself is fetched via JSON after the browser paints the shell.
  • Deeper pages (Show more reviews, pagination) require the session to carry the right cookies, and often a scroll interaction too.
  • Sort filters (newest, verified only) alter the request signature. Get one wrong and the marketplace serves you the default view with a subtle rate limit on top.
  • The whole page sits behind bot defenses that inspect the request's low-level signature, the header order, and the timing before they even look at the User-Agent.

Fetch that page with a plain HTTP request and you get the empty shell (no reviews, no verdict). Fetch it with a headless browser you set up yourself, and you inherit the ongoing bill of maintaining that browser, rotating exits, and patching against new detection rules every time a marketplace ships an anti-bot update.

The Solution

The pattern that works is a browser-based fetch you can call as a single API endpoint. You hand it the review URL. It renders the page, executes the JavaScript, waits for the review payload to hydrate, and hands you the DOM back. The session details come with the response, so the next paginated fetch can reuse them.

With FourA's Browser product, that reads like this in Python:

import requests

def fetch_reviews(url, api_key, sticky_proxy=None):
    body = {
        "url": url,
        "unblocker": True,
    }
    if sticky_proxy:
        body["proxy"] = sticky_proxy

    r = requests.post(
        "https://api.foura.ai/api/browser",
        headers={"Authorization": f"Bearer {api_key}"},
        json=body,
        timeout=60,
    ).json()

    return {
        "status": r.get("status"),
        "html": r.get("body"),
        "cookies": r.get("cookies"),
        "sticky_proxy": r.get("proxy"),
        "defense_solved": r.get("defenseSolved"),
    }

Two details matter for review scraping.

First, unblocker: true handles the wire-level defenses most marketplace review pages sit behind. Without it, you get a challenge page instead of reviews.

Second, the returned proxy value is the base36 id of the exit that got the successful render. Feed it back on the next call for the same product and the pagination will treat you as the same visitor. So the difference between page 1 loading fine and page 3 quietly returning nothing often comes down to whether you pinned the session.

For products where you only need a fresh weekly snapshot of the top 20 reviews, skip the stickiness and let each call pick its own exit. Save the pinning for the SKUs where deep pagination actually matters.

Results You Can Expect

A team running review aggregation on this pattern should think in these terms:

  • Coverage: every product's reviews from every marketplace where you sell, refreshed on whatever cadence your downstream needs, from daily for CX alerting to weekly for trend analysis.
  • Depth: first 3-5 pages of reviews per SKU per marketplace, which is where the actionable signal lives. The tail past page 5 is mostly noise for sentiment purposes.
  • Freshness: new reviews land in your sentiment pipeline within a day of being posted, not two weeks later once a monthly export runs.
  • Cost predictability: you pay per rendered fetch, so the budget scales with the number of SKUs you care about, not with the size of the team you'd need to maintain the browsers.

Those aren't magic numbers. They're what you get when the rendering half of the problem is off your team's plate.

If you're comparing this to a full marketplace monitoring stack, reviews aggregation is the sentiment-heavy cousin. Same rendering muscle, different downstream.

Key Takeaway

Every dashboard your brand looks at ends in a chart. But the chart is downstream. The upstream cost (the reason most brands don't do this well) is the rendering fleet: keeping 15 marketplace surfaces reachable, kept alive against defenses that were built to keep you out, at a price that isn't three engineers plus a proxy contract.

If the rendering half is what's blocking your review aggregation project, that's the piece to buy. The sentiment model, the dashboard, the alerting — those are the parts your team can build in an afternoon once fresh reviews start landing.