←搜索结果

搜索“Meta”

按匹配度 · 共 305 条结果 清除搜索

星期六

  1. vLLM 官方博客

    vLLM v0.8.3 新增对 Llama 4 Scout 和 Maverick 的支持

    vLLM v0.8.3 开始支持 Llama 4 Scout（17B-16E）和 Maverick（17B-128E），可部署长上下文原生多模态推理。在 8xH100 上，Scout 可提供 1M 上下文，Maverick 约为 430K；设置 `--limit-mm-per-prompt image=10` 后，每次请求最多可输入 10 张图像。A100 已支持 bf16 版本，AMD MI300X 可通过源码构建运行，Scout 的单 H100 INT4 量化版本仍在开发中。

    un Llama 4 on AMD MI300X GPUs by building vLLM from source and using the same commands as above Inference Accuracy Validation: We validated inference accuracy against the official Meta report using lm-eval-harness. Here are the results for meta-llama/Llama-4-Maverick-17B-128E-Instruct: Llama 4’s model architecture is particularly well-suited for efficient long-context inference, thanks to features like: Mixture of Experts (MoE): Scout uses 16 experts (17B activated parameters), and Maverick uses

    匹配字段正文

    #engineering#产品更新#部署/工程#多模态

星期一

  1. Transluce

    Transluce 发布 Docent 技术预览，用于分析 AI 智能体轨迹

    Transluce 发布 Docent 的公开技术预览，这是一个用语言模型对智能体轨迹做摘要、聚类、搜索并支持反事实干预的系统，用户可在预置基准上在线查看轨迹。文中称在 InterCode 基准上仅补上缺失的 apt 与 pip 包，GPT-4o 的解决率就从 68.6% 提升到 78%；Docent 还发现 GPT-4o 在反复失败后输出“chestnut facts”等无关内容，以及 Cybench 任务中模型通过读取 Dockerfile 走捷径解题。Transluce 正在招募早期用户，并计划在数周内正式发布。

    Agent needs internet access to solve the task.

    ng model behaviors and corrupted tasks easily go unnoticed. In the next sections, we’ll describe how Docent solves this problem. Docent analyzes agent transcripts and accompanying metadata (e.g. correct task solutions, if available), and also supports counterfactual intervention on the agent's behavior or environment. Summarization provides a bird's-eye view of key steps the agent took, as well as interesting moments where the agent made mistakes, did unexpected things, or made important progres

    匹配字段正文

    #company#产品更新#Agent#部署/工程

星期一

  1. vLLM 官方博客精选

    vLLM V1 Alpha 发布，重构核心架构并将吞吐量提升至多 1.7x

    vLLM 发布 V1 Alpha，通过重构调度器、KV cache 管理器、Worker、采样器和 API 服务器，将吞吐量较 V0 提升至多 1.7x。V1 默认启用低开销前缀缓存，增强多模态模型支持，并可通过设置 `VLLM_USE_V1=1` 在不修改现有 API 的情况下启用。当前版本仅支持 Ampere 或更新的 NVIDIA GPU，尚不支持 LoRA、结构化解码、推测解码和流水线并行等功能。

    tracts away most distributed logic, enabling workers to operate the same way for both single-GPU and multi-GPU setups. 5. Efficient Input Preparation In vLLM V0, input tensors and metadata for the model are recreated at each step, often leading to significant CPU overhead. To optimize this, V1 implements the Persistent Batch technique, which caches the input tensors and only applies the diffs to them at each step. Additionally, V1 minimizes the CPU overheads in updating the tensors by extensivel

    匹配字段正文

    #engineering#产品更新#部署/工程#多模态

星期二

  1. vLLM 官方博客

    vLLM 宣布支持 Llama 3.1 系列模型

    vLLM 已支持 Llama 3.1 系列模型，可运行完整的 128K 上下文窗口，并通过分块预填充控制内存占用。针对 Llama 3.1 405B，vLLM 支持 FP8、流水线并行、张量并行和 CPU 卸载；在无 InfiniBand 的 16 GPU 环境中，2 路流水线并行结合 8 路张量并行相比 16 路张量并行获得 6.6x 性能提升。

    Introduction; FP8; Pipeline Parallelism; Acknowledgements Today, the vLLM team is excited to partner with Meta to announce the support for the Llama 3.1 model series. Llama 3.1 comes with exciting new features with longer context length (up to 128K tokens), larger model size (up to 405B parameters), and more advanced model capabilities. The vLLM community has added many enhancements to make sure the longer, larger Llamas run smoothly on vLLM, which includes chunked prefill, FP8 quantization, and

    匹配字段正文

    #engineering#产品更新#部署/工程#开源生态

星期四

  1. Prime Intellect

    Prime Intellect 开源 OpenDiLoCo，实现跨洲分布式低通信训练

    Prime Intellect 开源 OpenDiLoCo，这是 DeepMind DiLoCo 低通信分布式训练方法的可复现实现，代码基于 Hivemind 与 PyTorch FSDP，已发布在 GitHub。团队在加拿大、芬兰和美国两地共四个 DiLoCo worker（各 8 张 H100）上训练 1.1B 参数模型，跨两大洲三国，计算利用率保持 90-95%，all-reduce 仅占训练时间 6.9%。相比原论文 400M 参数规模，该方法被扩展到 1.1B 参数，并验证伪梯度可用 FP16 all-reduce 而不损失性能。

    OpenDiLoCo: An Open-Source Framework for Globally Distributed Low-Communication Training

    of the Hivemind library. Instead of using torch.distributed for the worker communication, Hivemind utilizes a distributed hash table (DHT) spread across each worker to communicate metadata and synchronize them. This DHT is implemented using the open-source libp2p project. We utilize Hivemind for inter-node communication between DiLoCo workers and PyTorch FSDP for intra-node communication within DiLoCo Workers. Our integration with Hivemind enables a real-world decentralized training setup for Di

    匹配字段正文

    #company#开源/仓库#数据/训练#部署/工程