---
title: "DeepSeek open-sources DeepInfer, the engine that serves its V4.1 Flash model"
url: https://anawords.com/story/deepseek-deepinfer-open-source-inference-framework
published: 2026-10-22T20:19:16+05:30
updated: 2026-10-22T20:19:16+05:30
section: Open Source AI
author: Mayank Sahu
checked: sources checked by AI, approved by editor
version: 1
---

# DeepSeek open-sources DeepInfer, the engine that serves its V4.1 Flash model

The Apache 2.0 framework runs on Nvidia Blackwell GPUs and Huawei Ascend 950 chips. DeepSeek kept some parts private, including cache management and load balancing.

**In short:** DeepSeek released DeepInfer on 20 October 2026, the inference framework it uses to serve its models, as open source under the Apache 2.0 licence. It currently supports only DeepSeek-V4.1-Flash and runs on Nvidia SM100 Blackwell GPUs and Huawei Ascend 950 chips.

## What DeepSeek released

DeepInfer is the software DeepSeek uses in production to run its AI models for users, now published on GitHub under the Apache 2.0 licence.[^1][^2] It supports one model for now, DeepSeek-V4.1-Flash, and DeepSeek says future models will be added.[^1][^2] The project is built mainly on vLLM, with ideas from SGLang and Nvidia’s TensorRT-LLM.[^1][^2]

## What is inside

- **Speculative decoding:** a small draft model proposes several tokens at once and the main model checks them in one pass.[^1][^2]
- **Faster prompt processing:** a technique that, DeepSeek says, nearly halves prefill computation.[^1]
- **Splitting work across chips**, plus features for consistent outputs and fixed output formats.[^2]
- **deepseek-recipe:** a companion library that converts text to tokens and back, matching DeepSeek’s API.[^1][^2]

## What was left out

DeepSeek did not release its cache management, separation of prompt processing from answer generation, or load-balancing code, which it says is tightly tied to internal systems.[^1][^2]

## Hardware

It runs on Nvidia SM100 (Blackwell) data-centre GPUs and Huawei Ascend 950 chips.[^1][^2] Older Nvidia architectures such as Hopper are not listed.[^1]

## Performance

DeepSeek reports a 4% to 7% throughput gain at production load from its decoding changes.[^1] SiliconANGLE reports DeepSeek disclosed revenue of $38 per GPU per hour at off-peak prices.[^1] These are DeepSeek’s own figures; no independent tests had been published.[^2] By the afternoon of release, the repository had more than 8,000 GitHub stars, TNW reports.[^2]

## What it means

Most open-source AI releases share model weights. Sharing the serving engine is less common and very practical: it shows how a lab that sells cheap API access keeps its costs down. Cloud providers and companies running DeepSeek models on their own servers can now use the same methods. The vLLM project said it would apply ideas from DeepInfer.[^2]

Support for Huawei’s Ascend chips is notable too, giving teams a tested option outside Nvidia hardware. For Indian companies hosting open models on their own GPUs, the parts that matter most for large-scale serving, caching and load balancing, still need other tools.

Separately, the US recently ordered federal civilian agencies to remove DeepSeek products; that order does not apply to private companies.

## What to watch

- Independent benchmarks of DeepInfer against vLLM and SGLang.
- Whether DeepSeek adds support for more models or Nvidia Hopper GPUs.

Related: [US Homeland Security orders DeepSeek removed from federal civilian systems](/story/dhs-orders-deepseek-removed-federal-civilian-agency-systems), [Google’s TorchTPU](/story/google-torchtpu-open-source-pytorch-on-tpus) and [DeepSeek’s funding round](/story/deepseek-funding-round-doubles-tencent-catl-ipo-report).

## Fact ledger

- [Reported] DeepSeek open-sourced DeepInfer under Apache 2.0 on 20 October 2026. (source 1,2)
- [Reported] It supports only DeepSeek-V4.1-Flash for now. (source 1,2)
- [Reported] It runs on Nvidia SM100 Blackwell GPUs and Huawei Ascend 950 chips. (source 1,2)
- [Reported] Cache management and load-balancing code were not released. (source 1,2)
- [Company claim] DeepSeek reports a 4% to 7% throughput gain at production load. (source 1)
- [Not known] Independent performance results.

## Sources

1. [SiliconANGLE: DeepSeek open-sources DeepInfer, the inference framework behind its production services](https://siliconangle.com/2026/10/20/deepseek-open-sources-deepinfer-the-inference-framework-behind-its-production-services/)
2. [TNW: DeepSeek open-sources DeepInfer, the engine that runs its AI models](https://thenextweb.com/news/deepseek-deepinfer-open-source-inference-engine)