DeepSeek and Peking University Open-Source DSpark: New Framework Boosts LLM Inference Speed by up to 85%

DeepSeek has partnered with Peking University to release DSpark, a new inference acceleration framework designed to eliminate performance bottlenecks in high-concurrency production environments for large language models (LLMs). The framework, now deployed in the preview versions of DeepSeek-V4-Flash and DeepSeek-V4-Pro, achieves a 60% to 85% improvement in single-user generation speeds compared to the previous MTP-1 baseline, without sacrificing output quality.

The announcement, made on June 27, includes the full open-sourcing of the framework's training code, evaluation scripts, and model checkpoints via the DeepSpec project on GitHub.

Solving the Autoregressive Bottleneck

LLMs traditionally generate text token-by-token in an autoregressive manner, requiring a complete forward pass for each new token. This causes inference latency to grow linearly with output length, resulting in the sluggish response times common in AI chat systems.

Speculative decoding offers a solution by using a lightweight "draft" model to rapidly generate candidate token sequences, which are then verified in parallel by the full-scale target model. Accepted tokens conform strictly to the target distribution, ensuring no quality loss while accelerating generation. However, existing approaches face a trade-off:

• Autoregressive draft models (like Eagle3) generate tokens sequentially, offering strong dependency modeling and high acceptance rates, but their latency increases with candidate length, forcing the use of short candidate blocks.
• Parallel draft models (like DFlash) generate all candidates in one forward pass, eliminating length-dependent latency, but suffer from declining acceptance rates in later positions due to the lack of intra-block dependencies. This leads to wasted computational resources during verification, particularly harmful in high-concurrency scenarios where batch processing capacity is precious.

DSpark's Dual Innovation

DSpark introduces two complementary mechanisms to break this deadlock.

Semi-Autoregressive Draft Generation: The framework employs a hybrid architecture featuring a computationally intensive parallel backbone (improved from DFlash) that generates hidden states and base logits for all candidate positions simultaneously. A lightweight sequential module then injects prefix dependency information token-by-token. This module offers two implementations: a Markov head (depending only on the previous token) and an RNN head (accumulating full prefix information through recurrent states).

Testing shows that a two-layer Transformer depth DSpark outperforms a five-layer DFlash in acceptance length across all domains, demonstrating that introducing limited autoregressive dependencies is more parameter-efficient than simply stacking parallel layers.

Confidence-Based Scheduling Verification: DSpark assigns a confidence score to each candidate position, predicting the survival probability of that token conditioned on all previous tokens being accepted. After calibration against empirical acceptance rates, a hardware-aware prefix scheduler treats verification length selection as a global throughput maximization problem. Given a batch of concurrent requests and their position-level confidences, the scheduler dynamically determines how much of each candidate prefix to verify, prioritizing computational resources for tokens with the highest global survival probability.

Production Performance

In offline benchmarks using Qwen3 (4B/8B/14B) and Gemma4-12B as target models, DSpark outperformed both Eagle3 and DFlash across mathematical reasoning, code generation, and dialogue tasks. For Qwen3-4B, DSpark improved average acceptance length by 30.9% over Eagle3 and 16.3% over DFlash.

In live production environments, the gains were substantial:

• On DeepSeek-V4-Flash, when maintaining a minimum speed of 80 tokens/s per user, DSpark increased aggregate throughput by 51%. When the Service Level Agreement (SLA) tightened to 120 tokens/s—a threshold where the MTP-1 baseline neared its operational limit—DSpark achieved a 661% throughput advantage while maintaining viable batch processing.
• On DeepSeek-V4-Pro, throughput improved by 52% at 35 tokens/s SLA, and by 406% at 50 tokens/s SLA.

The scheduler demonstrates adaptive load balancing, allocating 4-6 tokens for verification during low-concurrency periods to utilize idle resources, while smoothly reducing verification length as concurrency rises to prevent resource contention.

Open Source and Limitations

The complete DSpark implementation, alongside DFlash and Eagle3 baselines, is available on the DeepSpec GitHub repository and Hugging Face.

The researchers note one current limitation: even when the scheduler truncates suffix tokens due to low confidence, the parallel backbone must still generate the full initial candidate block. For complex queries with inherently low acceptance rates, this draft computation overhead cannot be recovered.