←搜索结果

搜索“DeepSeek”

按匹配度 · 共 123 条结果 清除搜索

星期一

  1. vLLM 官方博客

    MiniMax-M1 混合架构如何借助 vLLM 实现长上下文快速推理

    vLLM 通过内存管理与底层计算优化支持 MiniMax-M1 的混合架构，实现长上下文高效推理与部署。MiniMax-M1 总参数量为 456B，推理时动态激活约 45.9B 参数；其 Lightning Attention 已通过 Triton 集成至 vLLM。文章还提供了使用 vLLM 和 Docker 部署该模型的快速指南。

    oftware engineering, tool use, and long-context understanding. MiniMax-M1 leads among open-source models; Right: Theoretical inference FLOPs scaling with token length. Compared to DeepSeek R1, MiniMax-M1 uses only 25% of the FLOPs when generating sequences of 100k tokens Deploying MiniMax-M1 with vLLM We recommend deploying MiniMax-M1 using vLLM for optimal performance. Our tests demonstrate the following key benefits: Outstanding throughput; Efficient and intelligent memory management; Robust s

    匹配字段正文

    #engineering#教程/实践#部署/工程#推理

星期一

  1. vLLM 官方博客

    vLLM 分布式推理中的张量并行与流水线并行

    vLLM 介绍了如何结合张量并行和流水线并行，在多 GPU 或多节点上部署超出单卡容量的大模型。张量并行可降低延迟但依赖高带宽互连，流水线并行通信开销较低且更适合跨节点部署。案例中从 TP=1 扩展到 TP=2 后，KV cache block 增加 13.9x，token 吞吐量提升 3.9x。

    idth interconnects between each GPU, like NVLink or InfiniBand, to minimize overhead from the increased communication costs. Pipeline Parallelism For extremely large models (e.g., DeepSeek R1, Llama 3.1 405B), a single node may not suffice. Pipeline parallelism shards models across nodes, each handling specific contiguous model layers. How It Works Each GPU loads and processes a distinct set of layers; Send/Receive Operations: Intermediate activations are transmitted between GPUs as computation

    匹配字段正文

    #engineering#教程/实践#部署/工程#推理

星期二

  1. vLLM 官方博客

    vLLM 结构化解码入门与 XGrammar 集成解析

    vLLM 介绍结构化解码的原理与实现，并已支持 Outlines 和 XGrammar 两种后端，以约束大语言模型输出符合指定结构。XGrammar 集成在负载下可将每个输出 token 的生成时间改善至多 5x，不适用的请求会回退至 Outlines。规划中的 v1 将重点优化性能，并探索调度器级掩码广播以支持混合请求批处理。

    top level view of structure decoding

    o the same mechanism of "using a format for the model to structurally sampling outputs.” ↩; See this blog post from HuggingFace for using logit processors to control the generation process. ↩ Related Posts DeepSeek-V4.1-Flash on vLLM: 5x Agentic Throughput Since Day 0 Within three weeks of release, vLLM made DeepSeek-V4.1-Flash 1.9x faster at low concurrency and lifted its throughput 5x on SemiAnalysis AgentX, with SWA bounded replay, CUDA graphs, DeepSeek's new kernels, and vLLM kernel fusions.

    匹配字段正文

    #engineering#教程/实践#部署/工程#Agent