蚂蚁 inclusionAI 发布 vllm-ling-v3 仓库
蚂蚁 inclusionAI 在 GitHub 上线 vllm-ling-v3 仓库,基于 vLLM 提供 LLM 推理与服务。vLLM 支持 PagedAttention、连续批处理、FP8/INT4 等量化,并兼容 200+ Hugging Face 模型架构,覆盖 NVIDIA、AMD、Intel GPU 及华为昇腾等硬件。
aX GPU, and more vLLM seamlessly supports 200+ model architectures on Hugging Face, including: Decoder-only LLMs (e.g., Llama, Qwen, Gemma); Mixture-of-Expert LLMs (e.g., Mixtral, DeepSeek-V3, Qwen-MoE, GPT-OSS); Hybrid attention and state-space models (e.g., Mamba, Qwen3.5); Multi-modal models (e.g., LLaVA, Qwen-VL, Pixtral); Embedding and retrieval models (e.g., E5-Mistral, GTE, ColBERT); Reward and classification models (e.g., Qwen-Math) Find the full list of supported models here. Getting St
匹配字段正文
