←Search results

Search “Reasoning”

By relevance · Total 3272 results Clear search

Tuesday

  4. Ma Dongxi NLP

    DeepSeek Open-Sources Large-Scale Inference Framework DeepInfer Under Apache-2.0 License

    DeepSeek has open-sourced its high-performance large-scale inference framework DeepInfer under the Apache-2.0 license. The framework supports optimized kernels, model and sampler runtime, parallelism strategies such as Attention DP and balanced CP, MoE EP plus EPLB, as well as DSpark and confidence-scheduled verification, SWA bounded replay, asynchronous scheduling with a zero-overhead batch scheduler, batch-invariant inference, deterministic sampling, and structured output. Ma Dongxi NLP reposted the news and said "Respect."

    DeepSeek has open-sourced its high-performance large-scale inference framework DeepInfer under the Apache-2.0 license. The framework supports optimized kernels, model and sampler runtime, parallelism strategies such as Attention DP and balanced CP, MoE EP plus EPLB, as well as DSpark and confidence-scheduled verification, SWA bounded replay, asynchronous scheduling with a zero-overhead batch scheduler, batch-invariant inference, deterministic sampling, and structured output. Ma Dongxi NLP repost

    Matched fieldsTitle、Summary

    #hot-signal#English text is being prepared.#DeepSeek#Product updates
  5. Shao MengSelected

    DeepSeek Open-Sources High-Performance Inference Framework DeepInfer

    DeepSeek has open-sourced DeepInfer, the high-performance inference framework supporting its online services, under the Apache 2.0 license. This release focuses on offline inference, is built primarily on vLLM, and draws on SGLang and TensorRT-LLM. It integrates previously separately open-sourced kernels such as FlashMLA, DeepGEMM, DeepEP, and EPLB into a complete engine, filling in system layers such as scheduling, parallelism, and speculative decoding, while supporting NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs. The original article also breaks down designs including DSpark speculative decoding, YoCo SWA bounded replay, asynchronous scheduling, batch-invariant deterministic inference, and regex/JSON schema/EBNF structured output.

    DeepSeek has open-sourced DeepInfer, the high-performance inference framework supporting its online services, under the Apache 2.0 license. This release focuses on offline inference, is built primarily on vLLM, and draws on SGLang and TensorRT-LLM. It integrates previously separately open-sourced kernels such as FlashMLA, DeepGEMM, DeepEP, and EPLB into a complete engine, filling in system layers such as scheduling, parallelism, and speculative decoding, while supporting NVIDIA SM100 GPUs and Hu

    Matched fieldsTitle、Summary、Body

    #hot-signal#English text is being prepared.#Reasoning#English text is being prepared.
  6. Tianyi CuiSelected

    DeepSeek open-sources the large-scale inference framework DeepInfer under the Apache-2.0 license

    DeepSeek has open-sourced the high-performance large-scale inference framework DeepInfer under the Apache-2.0 license. The framework supports optimized kernels, model and sampler runtimes; on parallelism it provides Attention DP, balanced CP, and MoE EP + EPLB, and includes DSpark and confidence scheduling validation, SWA bounded replay, asynchronous scheduling of a zero-overhead batch scheduler, batch-invariant inference, deterministic sampling, and structured output.

    Matched fieldsTitle、Summary

    #hot-signal#English text is being prepared.#Reasoning#English text is being prepared.
  13. Tech.eu

    Former Google DeepMind researcher founds London AI company Metis Reasoning, focusing on compute-efficient reasoning

    Metis Reasoning, a London AI company co-founded by former Google DeepMind research lead and AlphaGo co-creator Thore Graepel, has officially launched, seeking a compute-efficient path distinct from the frontier models of Anthropic and OpenAI, and is believed to have raised tens of millions of dollars in initial funding. The company has a team of 12 people, including researchers, mathematicians, robotics experts, and technology entrepreneurs. It believes that LLMs have fundamental limitations in reasoning, that results are difficult to verify, and that there is a lack of audit trails; its approach is more verifiable, more robust, and safer, and it says it has already achieved some positive results on frontier reasoning problems at a cost far below that of frontier models. Metis has not yet announced a release date for its first AI model.

    Metis Reasoning, a London AI company co-founded by former Google DeepMind research lead and AlphaGo co-creator Thore Graepel, has officially launched, seeking a compute-efficient path distinct from the frontier models of Anthropic and OpenAI, and is believed to have raised tens of millions of dollars in initial funding. The company has a team of 12 people, including researchers, mathematicians, robotics experts, and technology entrepreneurs. It believes that LLMs have fundamental limitations in

    Matched fieldsTitle、Summary、Body

    #hot-signal#Industry news#Reasoning#Agents

Monday

Sunday

  1. Shao Meng

    Stanford CS336 Lecture 10: Percy Liang on Inference Optimization

    Stanford CS336 Lecture 10 "Inference" is taught by Percy Liang, who uses a single mathematical thread to connect three layers of inference optimization—architecture, algorithms, and systems: at the architecture layer, GQA/MLA/CLA reduce dimensionality, local attention truncates context, and quantization, pruning, and distillation compress the KV cache and parameters; at the algorithm layer, speculative sampling achieves lossless acceleration; at the systems layer, continuous batching, paging, prefix sharing, and copy-on-write are introduced. The lecture notes are in executable form, with formulas derived step by step by sympy and verified with assertions. Percy believes that the KV cache is the structural root of the Transformer's unfriendliness to inference, and that architectures natively designed for inference (SSMs, linear attention, diffusion-style non-autoregressive generation) may offer the greatest room for improvement.

    Stanford CS336 Lecture 10 "Inference" is taught by Percy Liang, who uses a single mathematical thread to connect three layers of inference optimization—architecture, algorithms, and systems: at the architecture layer, GQA/MLA/CLA reduce dimensionality, local attention truncates context, and quantization, pruning, and distillation compress the KV cache and parameters; at the algorithm layer, speculative sampling achieves lossless acceleration; at the systems layer, continuous batching, paging, pr

    Matched fieldsTitle、Summary、Body

    #hot-signal#Tutorials / practice#Reasoning#English text is being prepared.