Insurance comparison sites expose weakness in legacy technology

Online price comparison shoppers push legacy technology to its limits

With up to 52 per cent of us now using comparison web sites to obtain insurance quotes, according to estimates, it is no surprise that many insurance company web sites are struggling to cope with the extra traffic being sent their way.

The favoured approach to a back-end system performance problem which ends in quotation engine search results taking minutes rather than seconds to display is to increase its processing power. But upgrading the hardware may not be the most cost-effective solution in every case.

Automated requests from comparison sites are usually sent to individual company web sites via XML data streams, which query information stored in legacy databases such as DB2 and Oracle on mainframe computers running Z/OS using the CICS transaction server. These systems and application code were originally designed 20 or 25 years ago to be accessed by that company’s own sales and service staff.

When insurance companies first entered the online age, they built their web front ends to give prospective customers direct access to the same data so they could retrieve their own quotes. But the sheer volume of automated requests forwarded from heavily advertised comparison web sites such as Gocompare, Compare the Market and Confused.com is now swamping those systems, with many seeing a tenfold increase in the demands placed on available processing power.

Philip Mann, principal consultant at application performance software vendor and consultancy Macro 4, has already encountered companies struggling with exactly this problem.

“They would not want me to name them, but we do have a couple of customers in that business which are aware of the situation, although that is not to say we are actively talking to them about what they can do about it, which is a shame,” he said.

“One has a mainframe database for car insurance, which was then duplicated for its call centre operation, and again with a wrapper around it to be used as the back end for its comparison web site system. So they now have three systems with all the original performance issues and goblins in them.”

Insurance comparison sites are not the only companies that can suffer from performance bottlenecks within back-end databases – any organisation handling large volumes of online transactions is susceptible to similar problems.

In most cases, they tend to upgrade the server hardware hosting those databases and transaction servers, taking the view that hardware able to process more million instructions per seconds (MIPS) will quickly improve matters.

This is often true, but given the price of new mainframe hardware from IBM, and x86-based high-performance computing clusters from a range of other vendors, it is not always the most cost-effective approach.

"Sometimes, chucking more processing power works to a certain extent, but people rarely say instead of using more resource to do this, should we look at it from an application processing point of view and see if we can isolate errors where resources are being used and make it more efficient," said Mann.

"The great danger is wasting time tuning something that was not running that badly in the first place, instead of focusing on finding out which processes or jobs are being used where, and what sort of service individual applications are getting from the system."

Macro 4 is just one of many companies supplying application performance monitoring software that identifies hot spots in application code to help reduce batch request response times. According to research firm Gartner, others include HP, CA, Quest Software, Compuware, IBM (Tivoli), Oracle, Nimsoft, Opnet and Progress Software.