LMSYS 公开 infer-forge:围绕 SGLang 的 Harness、Loop 与 Graph 工程方法
LMSYS 发布 infer-forge 方法论文,介绍围绕 SGLang 的推理工程系统,由 MonoRepo、Harness、Task Loop 和 Task Graph 四层结构组成,用于让跨仓库的推理优化工作可复现、可验证。该系统为内部工程系统,并非 SGLang 或 LMSYS 官方组件,目前不开源,文章公开的是构建方法论,供团队交给 AI 编码工具并结合自身系统上下文搭建适配实现。文中给出使用记录:一位工程师 4 月至 7 月的在飞任务峰值从 2 个升至 9 个,DeepSeek-V4-Pro 服务项目中 38 个可独立验证的任务节点被组织成任务图。
le are stable enough to guide future work. Standardization follows proven practice; it does not attempt to predict every path in advance. 4.2.2 Starting Context Starting Context draws the line between known state and assumption. A Task may begin from a blank state, a prepared environment, or an upstream result. What matters is that the source and verification status of that state remain explicit. Custom Setup records conditions prepared for the current Task, such as the model, container image, d
匹配字段正文



