Search “Reasoning”

English translation is being prepared.

DeepSeek open-sources inference framework DeepInfer and codec components under Apache 2.0

DeepSeek has open-sourced, under the Apache 2.0 license, the inference framework DeepInfer that powers its online services and the API-identical codec component deepseek-recipe, and the company says inference code for future new models will also continue to be merged into this repository. The framework includes high-performance operators, distributed inference, speculative decoding, SWA Bounded Replay, and structured output, but the README shows that this release only includes the offline inference engine; the KV cache system, prefill-decode disaggregated deployment, and multi-replica load balancing are not included. On hardware, it currently supports only NVIDIA's Blackwell architecture SM100 and Huawei Ascend 950, and the only supported model is V4.1-Flash.

English translation is being prepared.

English translation is being prepared.

Zyphra Inference Wave 2: Accelerating MoE and Long-Context Inference Serving on MI355X

Zyphra has released ZOE, its second-generation inference optimization, which on AMD Instinct MI355X reduces GLM-5.3's time to first token (TTFT) at 512K context from 178.8 seconds to 64.6 seconds (2.8x), and from 42.3 seconds to 11.8 seconds (3.6x) at 90% prefix cache hit rate. Its CSA top-K selection HIP kernel raises DeepSeek-V4-Pro's single-user decode throughput at 1M context from 43.5 to 61.1 tokens/s (1.40x). AMD MXFP4 quantization of GLM-5.3 shrinks the model from 756 GB to 410 GB, with benchmark scores such as AIME25 within 3 points of FP8.

English translation is being prepared.

English translation is being prepared.

DeepSeek open-sources the inference framework DeepInfer and components such as deepseek-recipe

DeepSeek has open-sourced its in-house inference framework DeepInfer, and simultaneously released key components for deploying DeepSeek models, including deepseek-recipe. DeepInfer already supports large-scale serving of DeepSeek-V4.1-Flash in production, with a focus on high throughput and low latency. DeepSeek said that publishing its production stack is intended to help the community lower compute costs, reduce energy consumption, and enable more organizations to deploy DeepSeek models.

DeepSeek Open-Sources Internal Inference Framework DeepInfer

DeepSeek announced the open-sourcing of its internal inference framework DeepInfer, which supports large-scale operation of DeepSeek-V4.1-Flash and focuses on high throughput and low latency. DeepSeek said it is making this production stack public to help the community lower compute costs, reduce energy consumption, and make AI more affordable, and included a link to the repository. Yuchen Jin reposted the news and commented "Real Open AI."

English translation is being prepared.

English translation is being prepared.

DeepSeek open-sources its inference framework DeepInfer

DeepSeek announced that it is fully open-sourcing its self-developed inference framework DeepInfer, which is used to support large-scale, high-throughput, low-latency operation of DeepSeek-V4.1-Flash. The company said it is sharing its production stack to help the community lower compute costs and energy consumption and make AI more affordable. Nathan Lambert commented that this is a distinctive way to gain model adoption and is good for the community, and noted that the timing is delicate because the government has just banned DeepSeek in federal agencies.

DeepSeek open-sources inference framework DeepInfer and Token codec component deepseek-recipe

DeepSeek open-sources its self-developed inference framework DeepInfer (Apache 2.0) and the production-grade Token codec component deepseek-recipe (MIT), supporting DeepSeek V4.1 Flash in serving 155 trillion Tokens per day on a single cluster. Official online statistics show an average of 77K Tokens input per request, a time to first token of about 400 milliseconds, 230 Tokens output per second during the generation phase, and an average of 96% of input hitting the cache. DeepInfer currently supports V4.1 Flash; the open-source version provides only offline inference, and the KV cache management and load balancing code has not yet been fully open-sourced because it is tied to internal infrastructure.

English translation is being prepared.

English translation is being prepared.

English translation is being prepared.

English translation is being prepared.

English translation is being prepared.

DeepSeek Open-Sources Large-Scale Inference Framework DeepInfer Under Apache-2.0 License

DeepSeek has open-sourced its high-performance large-scale inference framework DeepInfer under the Apache-2.0 license. The framework supports optimized kernels, model and sampler runtime, parallelism strategies such as Attention DP and balanced CP, MoE EP plus EPLB, as well as DSpark and confidence-scheduled verification, SWA bounded replay, asynchronous scheduling with a zero-overhead batch scheduler, batch-invariant inference, deterministic sampling, and structured output. Ma Dongxi NLP reposted the news and said "Respect."

DeepSeek Open-Sources High-Performance Inference Framework DeepInfer

DeepSeek has open-sourced DeepInfer, the high-performance inference framework supporting its online services, under the Apache 2.0 license. This release focuses on offline inference, is built primarily on vLLM, and draws on SGLang and TensorRT-LLM. It integrates previously separately open-sourced kernels such as FlashMLA, DeepGEMM, DeepEP, and EPLB into a complete engine, filling in system layers such as scheduling, parallelism, and speculative decoding, while supporting NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs. The original article also breaks down designs including DSpark speculative decoding, YoCo SWA bounded replay, asynchronous scheduling, batch-invariant deterministic inference, and regex/JSON schema/EBNF structured output.