Study puts hard numbers on AI productivity boost

ChatGPT boosts speed by 25% and performance more than 40%

A study led by Harvard researchers has quantified the performance benefits of using AI at management consultancy Boston Consulting Group (BCG).

Findings from ‘Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality' revealed that consultants who harnessed generative AI demonstrated significant improvements in task completion frequency, speed and overall quality.

The lowest-performing consultants experienced the most substantial enhancements when integrating generative AI into their work.

A collaborative study

The study was conducted collaboratively by data scientists and scholars from universities including Harvard, Wharton and the Massachusetts Institute of Technology, to examine the practical application of generative AI from ChatGPT within the corporate world.

Harvard's Fabrizio Dell'Acqua, the paper's lead author, said: "The fact that we could boost the performance of these highly paid, highly skilled consultants, from top, elite MBA institutions, doing tasks that are very related to their everyday tasks, on average by 40%, I would say that's really impressive."

Cyborgs and Centaurs

The study further solidifies the notion that generative AI will have a profound impact on workforce productivity. However, it also provides cautionary findings on when and how not to use AI.

The paper identifies a ‘jagged technology frontier', a shifting and often challenging barrier that delineates tasks suited for AI and those beyond its current capabilities.

It highlights two emerging patterns of AI usage among technologically savvy consultants, termed ‘Cyborg' and ‘Centaur' behaviours.

'Cyborgs' were those who completely integrate their task flow with the AI and continually interact with the technology, while 'Centaurs' divide and delegate their solution-creation activities between themselves and the AI.

These approaches may serve as models for navigating tasks where AI's capabilities are uncertain.

Increased speed of tasks with AI

The study involved 758 consultants, representing 7% of BCG's consultant pool. For tasks that fell within the AI capabilities frontier, consultants using AI completed 12.2% more tasks on average and accomplished tasks 25% faster compared to those without AI support.

Additionally, the AI-equipped consultants produced results with 40% higher quality than a control group without access to such technology.

AI acted as a skill leveller, with consultants who initially scored lower in baseline performance experiencing the most significant improvement (43%) when using AI. However, the study also revealed that individuals who employed AI for tasks outside its proficiency were prone to errors, placing excessive trust in AI.

The report suggests that the future of work may involve a blend of human and AI capabilities, with 'Centaurs' and 'Cyborgs' serving as potential role models. These individuals seamlessly switch between AI and human tasks, leveraging the strengths of both.