复现 ICML 2,200 篇论文后,我们学到了什么
Hugging Face 与 alphaXiv 组织社区借助编码智能体复现了 2,226 篇 ICML 2026 论文,并公开实验日志、代码和产物。受检论文中,51% 至少有一项主张获独立验证,23% 至少有一项被证伪或存在争议;结果也显示,智能体会误判尺度效应或单位问题,可靠审查仍需人类引导。
eflating perplexity roughly threefold. The abstract's "3.1% quality cost for 50% cache reduction" becomes roughly 9.4% once corrected. 🚨 False falsifications. Sometimes we found flaws in an attempted reproduction. One logboook claimed dramatically, "the paper's method is 2x slower than the baseline"; this turned out to be an arithmetic bug in the reproduction : per-trajectory time compared against per-batch-of-50 time. Correctly normalized, the participant's own data confirms the paper's claimed
匹配字段正文




