Transaction Processing Performance Council working group to deliver AI benchmark

Standardisation is required in artificial intelligence

The Transaction Processing Performance Council (TPC) has decided that artificial intelligence is now established enough that it needs benchmarks, and has formed a new working group (TPC-AI) to get the job done.

The TPC is a non-profit organisation whose members include some of the biggest names in technology, like Cisco, HPE, Microsoft, Intel and VMWare. The body was founded to define transaction processing and database benchmarks, recently delivering benchmarks for IoT gateways and hyperconverged infrastructure.

The variety of systems and tools used in the AI space prompted the TPC's decision. It says, ‘As industry investment in AI continues to accelerate, and the number of AI platforms and tools continue to proliferate, vendors and customers increasingly require an impartial means of comparing performance and total cost of ownership across a widening array of hardware and software systems.'

"Artificial intelligence will become mainstream across industry," said working group chairman and Cisco USC CTO Raghunath Nambiar. "These workloads are extremely data intensive [and] adjacent to the core area of focus for the TPC. Our new AI working group's priority is to develop a set of standardised metrics to enable fair price and performance comparisons across an array of different hardware and software stacks. And we're actively encouraging industry experts and members of the research community to help us in this effort."

The working group has between six and 12 months to complete its task. Nambiar says that finding workloads to use will be easy considering the TPC's membership. However, hardware - which is extremely varied - will be more of a challenge.