All posts

Live odds at scale: how OddFeeds runs a 250M-request feed on FourA

How OddFeeds runs a live sports-odds feed at over 250 million requests and 50TB a month on FourA, paying only for successful requests.

OddFeeds runs a live sports-odds feed. It collects betting odds from dozens of sources in real time and delivers them as one continuously updated stream.

That stream feeds OddStorm, a sports arbitrage platform that reads the odds to find arbitrage opportunities. Arbitrage is unforgiving: a price that arrives a second late, or slightly wrong, does not just look bad. It turns into millions of incorrect arbitrages. So the feed has one hard requirement that never relaxes: stay fast and stay accurate, continuously, across every source.

Both belong to OddStorm.Company, a sports data and arbitrage group. This is how OddFeeds runs its pipeline on FourA.

More than 250 million requests a month. Around 50 terabytes of traffic. Re-scanned every one to two seconds. Zero credits for a failed request.

Scanning the market every one to two seconds

Odds move constantly, so OddFeeds re-reads them every one to two seconds across dozens of sources. At that cadence the numbers get large fast: more than 250 million requests a month, and around 50 terabytes of traffic. Many of those sources sit behind serious bot management, which is where most scraping setups start to bleed money.

Why the usual approach breaks here

The bill compounds at this volume, and it depends on how a tool charges. Some charge per attempt, so every block you hit is money spent for nothing, and at hundreds of millions of requests failed attempts alone can outweigh the useful spend. Others charge by bandwidth, which looks cheap until the traffic is large: at a rate of $0.30 per GB, 50 terabytes a month works out to $15,000 a month, or $180,000 a year, before a single block is counted. On top of either model, the hardest sources need a real browser and a clean network path to load at all, and doing that for every request is slow and expensive. A pipeline that reads the market every second cannot afford these taxes.

Pay only for what works

On FourA, a failed request costs zero credits. Only a delivered, valid response is charged. That single rule changes the shape of the bill: blocks, timeouts, and retries stop adding up, and the cost tracks the data OddFeeds actually receives. At this scale, moving the price of failure to zero is the difference between a predictable monthly cost and one that drifts with every source.

Use the lightest method that works

FourA does not send every request the same way. Easy sources are read directly by the lightweight request engine. When a source pushes back, the proxy endpoint with the unblocker turned on clears the large majority of anti-bot defenses on its own, without a browser. For most protected sources that is enough, and it stays cheap.

Only the hardest sources need the browser engine, and even then just once. It loads the site the way a real browser does and establishes a valid, cleared session. That session is then handed back to the lightweight engine, which reuses it to pull fresh odds thousands of times at a fraction of the cost.

So the expensive path is both rare and paid only once, then spread across a long run of cheap reads. The chart below is one of those harder sessions: a single costly request to establish it, then a long tail of inexpensive refreshes.

Credits per request across one sessionOne browser solve, then the request engine replays the cleared session.010203030 cr2 cr eachrequest 1later in the sessionBrowser solveCheap replayAverage cost / request

Auto, FourA's one-shot endpoint that takes a URL and returns content, is used for discovery when OddFeeds maps a new source. The production pipeline drives the core request, proxy, and browser endpoints directly, orchestrated to keep the per-record cost low.

Staying reliable when sources push back

Sources behave differently. Some serve odds only to certain countries. Some challenge every visit. FourA routes each request through an appropriate exit and the right engine, so the feed keeps flowing without a person retuning it per source. The goal is not cleverness for its own sake. It is a legitimate data pipeline that stays up and stays accurate while the sources around it keep changing.

The result

OddFeeds runs a live odds feed at more than 250 million requests and 50 terabytes a month, refreshed every one to two seconds, with a cost that scales with successful data rather than with attempts or blocks. Hard sources are solved once and read cheaply after that, and failures cost nothing. That is what lets a feed this large stay both fast and affordable.

Build your pipeline on the same infrastructure

FourA is the platform OddFeeds runs on: pay only for successful requests, solve hard sources once, and scale without your bill scaling with your failures. Start free or see pricing.