Betfair has to deal with automated trading applications, legitimate pieces of software that hit the site with hundreds of requests per second.

Chief technology officer Rorie Devine says the site needs shaping and mitigation capability to make sure it can manage traffic.

Networking specialist DataPower is, therefore, providing data shaping technology for Betfair.

‘We are very happy to take automated trading programs through our application programming interface, but every response to a request is a cost. We want to make sure that the traffic profile is within the bands of which we want to receive it – where it is economic,’ says Devine.

He says Betfair is less keen on screen-scraping robots, software which gathers and aggregates data from web sites and creates high volumes of traffic that affects servers.

‘We want to be able to look at what automated trading programs are trying to do, how often they are trying to do it, and at their usage pattern,’ says Devine.

He says they have nothing in common with denial of service (DoS) attacks, aimed at bringing a web site down.

‘They are mostly automated agents sitting in a tier one ISP. Such programs will not receive a 404 error message, though. The worst case is they may receive a slightly delayed response,’ says Devine.

Managing traffic is nothing new to Betfair. It already manages spikes, created by punters betting simultaneously, using Citrix Netscaler network appliances.

The organisation also handles applications requests with additional compression and load balancing, queuing each request in the pipeline to make sure that no one is turned away in peak traffic periods.

‘Not many sites can handle two billion page requests a week. We want to make sure we are getting value for money from our hardware,’ says Devine.

And by deploying data shaping technology, Betfair has analysed that it can push out new investment plans for some of its infrastructure.