DeepSeek claims new technique boosts LLM serving efficiency by up to 85%

DSpark module improves output speed making more efficient use of GPU resources

DeepSeek has documented a new inference acceleration framework that it claims increases the efficiency of how LLMs are run.

The company, which this month become China’s most valuable AI startup, says the intention is to improve how efficiently existing models run in production.

The DSpark framework has already been deployed on live traffic, with DSpark variants of V4-Flash and V4-Pro available on Hugging Face.

“DeepSeek-V4-Pro-DSpark is not a new model. It is the same checkpoint with an additional speculative decoding module attached,” the company explains.

According to DeepSeek, the deployed configuration ("DSpark-5") improves per-user generation speed - how quickly generated text is streamed by the model - by 60%-85% on DeepSeek-V4-Flash and 57%-78% on DeepSeek-V4-Pro, thanks to changes in inference architecture.

The increase in output streaming speed is achieved through a “speculative decoding” module which predicts and verifies multiple tokens simultaneously, increasing throughput and reducing latency. By making more efficient use of GPU resources, more customers can be served using the same hardware, potentially reducing the infrastructure cost of serving LLMs.

If DeepSeek’s claimed efficiency gains results can be reproduced in practice, cost-per-token could fall substantially at a time when many organisations are worried about their AI bills. In use, reasoning models and AI agents consume far more tokens than earlier chatbots.

See also: The era of cheap AI is ending – what comes next?

DeepSeek has released DeepSpec "a full-stack codebase for training and evaluating draft models for speculative decoding" under the open source MIT licence.

Although the initial deployment showcases DSpark with DeepSeek V4, the DeepSpec framework is described as a “general-purpose speculative-decoding platform” and includes configurations and support for models by other vendors including Alibaba’s Qwen and Google’s Gemma.

If DSpark proves to be generally transferable it could represent a significant step towards reducing the cost of hosting AI models.

As consultancy McKinsey notes: “AI’s next breakthrough may not be a smarter model but a cheaper token.”