DeepSeek open-sources the inference engine behind its production services

DeepSeek just open-sourced its inference engine

DeepInfer is the high-performance framework that powers DeepSeek's large-scale production serving, now released under Apache 2.0 for offline inference. It supports DeepSeek-V4.1-Flash with optimized kernels like FlashMLA, DeepGEMM, and DeepEP, plus attention DP, MoE expert parallelism with EPLB, DSpark speculative decoding, batch-invariant and deterministic inference, and structured outputs. It runs on NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs. A KV cache system, disaggregation, and cross-replica load balancing are not included.