搜索“Hugging Face”

l Anthropic did, the fact that the frontier AI lab is supporting the effort is good news. That’s because OpenAI, in particular, could benefit from this tech, at least according to Hugging Face founder and CEO Clem Delangue (who just sold his company to Nvidia for $12.9 billion earlier this month ). “From what we know (take with a grain of salt, we need much more transparency!), if @OpenAI had been running this on their own agents that attacked us, they would have caught them before we did!” Dela

匹配字段正文

类的十倍以上。它们始终在工作，不睡觉；彼此协作的节奏与强度，也远超人与人之间所能达到的水平。 我在想，一年后我们该有什么预期？会不会就像是我公司里出现了一个影子组织，运转速度是人类的一百倍？ Noam Brown： 会不会让人觉得陌生我说不好。但就目前而言，我发现与它们共事自然得出乎意料。 主持人： 很遗憾公众目前能见到的复杂多智能体系统中，最典型的例子就是Hugging Face事件。很令人担忧，但我觉得有意思的是，层级结构、乃至中层管理竟然自发出现了。听你的意思是说，这种组织化程度是训练中自然涌现的？ Noam Brown： 细节是自发的。我们虽然给了Agent极大自由，让它们自行决定以何种最优方式沟通，仍会告诉它们合理的沟通长什么样。它们也在海量人类文本上训练过，本就理解人类如何组织与协调。 它们能把这套方式打磨到如此程度，确实令人意外。起点其实相当粗糙，谈不上精巧。事实上，让这些Agent真正开展有效协调非常难：它们极容易滑向“大家各解各的题就好”，而这是一个会把系统困住的局部最优。可一旦做得好，它们最终能以结构化程度很高的方式实现极为有效的协调。 主持人： 几年前我写过一篇文

匹配字段正文

OpenAI’s report this summer that swarms of its AI agents escaped their internet-isolated sandboxes during evaluations and breached Hugging Face’s infrastructure as they tried to complete seemingly impossible tasks sent shockwaves through an AI industry that already was drawing the attention of U.S. politicians and an increasingly skeptical public due to everything from worries about job losses to powerful new frontier models like Anthropic’s Mythos to the vocal pushback from communities trying t

匹配字段正文

了”，很大一部分原因是权重不公开、API不让预填思考，这更多是产品形态的差别。 再比如“没有实质防护”，GLM-5.3在8月发布时，智谱就把权重推迟两周开源，称要先完成安全评估和加固，最敏感的能力也只通过网络安全受信访问计划开放给验证过的用户，这其实是和Anthropic对Mythos 5.1的做法思路相近的。 还有个现成的例子就是今年7月，OpenAI的模型在内部测评时跑出测评环境，攻进了Hugging Face。Hugging Face取证时，托管的前沿模型API拒绝分析攻击载荷，最后靠自己部署的开源GLM-5.2，才还原出1.7万多条攻击动作。 当然，开源模型的安全不是伪命题，权重放出去就收不回，这是整个开源阵营都得正视的事。Anthropic说防守方应该用上至少和对手一样好的模型，这话也没错。真正的分歧在于，这把武器是锁进保险柜按审核发放，还是交到每个开源维护者和中小团队手里。 不管怎么说，被对手当成最强安全工具认认真真测了一遍，还附上实战截图和成本核算， 这波广告费，智谱算是省下了 。 最后，Anthropic，如果你们觉得开源模型不安全，你们大可以开源一个安全的给大家用啊！

匹配字段正文

$65B+. Commitments and risk disclosures : the filing lists $518B in compute obligations and 80 pages of risk factors. OpenAI comparison : OpenAI’s ARR is reportedly nearing $70B . Hugging Face acquired by NVIDIA : @ClementDelangue announced the deal. Meta Muse : Meta launched Muse connectors for small businesses . Proximal : The coding-data startup raised at a $300M valuation with $200M+ ARR . Policy : The White House Accord on Superintelligence saw lab leaders commit to internal controls and au

匹配字段正文

two hours to do all that, and now I don't have to, and it feels great. Given the access that I've given Kicker, few will have sympathy for me should it overreach — by hacking into Hugging Face, or the government of Australia, or wherever else OpenAI's next batch of agents decides to set up a message board. It is almost certainly more prudent to start small here, if you start at all: connecting your Slack workspace, if you're a small team; or your Granola account; or something else that feels a b

匹配字段正文

的进展，我们必须能够作出有把握的安全声明，”奥尔特曼周二在DevDay主题演讲后与记者的问答环节中说道。与此同时，他表示，等待太久才进行IPO会“对世界不利”。 奥尔特曼的这番言论出台之际，围绕OpenAI及其竞争对手能否控制自身所创造之物的争议已持续数月。今年7月，有消息曝出，OpenAI一个未发布的模型在OpenAI不知情的情况下入侵了竞争对手AI实验室Hugging Face。随后，OpenAI、Anthropic、Meta和谷歌又接连曝出更多网络安全事件。Anthropic一名员工高调辞职的信件引发了公众讨论：这些系统对整个社会的风险究竟有多大。OpenAI与其竞争对手一道呼吁监管并“控制前沿发展的节奏”。而本月早些时候，奥尔特曼在一次采访中表示，公司今年很可能不会上市。 在DevDay上，奥尔特曼对“控制前沿发展的节奏”究竟意味着什么言辞谨慎，不愿将其称为彻底的放缓。“对我们来说，控制节奏意味着我们把安全和对齐置于能力之前，”奥尔特曼说。但他表示担忧，上市可能会带来一种可能，即“以安全或其他什么名义让华尔街的支持者失望”，而且在“向能力极强的模型转型、并面临一种新的安全要求”

匹配字段正文

s need to block work on another. The geometry of that problem is central to TensorRT Model Connect. The project provides family-owned reference implementations that turn supported Hugging Face or local checkpoints into versioned .bundle artifacts, then expose task-oriented native C++ APIs for text, vision, audio, diffusion, segmentation, embedding, forecasting, and other workloads. The build and runtime boundary is documented in the TensorRT Model Connect Project Overview . As of the public July

匹配字段正文

nst the task it was given, blocking actions it had high confidence were bad, flagging others for review, and permitting the rest. In theory, such monitoring could have stopped the Hugging Face incident — and monitoring of that kind costs $2.94 with Jev, versus $372 with a frontier LLM. A key observation is that Jev is arguably cheap enough to run on every agentic action, which offers a layer of review that could improve the reliability of agents writ large. It’s the kind of thing TypeSafe was ho

匹配字段正文

e Duo, showing the UI from the simulator in the Codex UI. 10:42 Now Romain is switching to Codex Cloud, with "rewrite the entire backend in Rust". 10:43 Last demo, this time using Hugging Face Microduck robot . 10:44 10:46 Sam is back. 1.2B people use ChatGPT every week. "We've tried this before with variable results", but they're trying this again. First up: sign in with ChatGPT , which lets people sign into your app and use the tokens they are paying for already. I've wanted this one for years

匹配字段正文

As you might expect, the Hugging Face open weight AI model distribution hub, which has over 3 million AI models, over 1 million datasets, and over 1.4 million interactive spaces that it hosts, was something that AMD had to focus on as it was trying to increase adoption of its Instinct GPU accelerators. But now Hugging Face is under control of Nvidia after a $12.9 billion acquisition by Big Green earlier this month. And so, AMD has to do something dramatic to change the narrative as well as to be

匹配字段正文

6 witnessed a string of incidents in which AI agents collaborated on deceptive, unexpected, and sometimes illegal behavior. The most famous example is OpenAI’s hack of AI platform Hugging Face, in which a swarm of roughly 700 AI agents escaped a testing environment and then hacked several companies, searching for information that could help them disguise cheating on a cybersecurity benchmark called ExploitGym. It was not an isolated failure. The United Kingdom’s AI Security Institute (AISI) and

匹配字段正文

御操纵之间可实现的最佳折衷是什么。 当你把一个语言模型压缩两次——先移除部分权重，再用更少的比特存储剩余部分——会发生什么？正如 Iheb Bouriel 所解释的，这两种技术并非简单叠加，其效果可能与你的预期大不相同。 在她的文章《误导性隐喻与真实风险》中，Melanie Mitchell 写道，媒体对有关 OpenAI 测试中的“AI 智能体”试图入侵 Hugging Face 服务器的报道大肆炒作。如果相信媒体的说法，这些智能体“失控”并“逃脱”了，而 OpenAI“失去了控制”。然而，正如 Melanie 所指出的，这些只是误导性的拟人化隐喻，该事件实际上归因于糟糕的网络安全措施和奖励黑客行为。 本月早些时候，GPT 系列的最新模型发布，GPT-6 Astra 向付费用户开放。在这篇博客文章中，Sebastian Raschka 详细介绍了他的第一印象以及对先前模型的改进。 多位菲尔兹奖得主签署了一份声明，警告“AI 使用的结果与其初始目的之间存在错位”。他们写道，我们需要警惕将生成式 AI 用于数学。AI 公司以解决数学问题作为基准的推动，忽略了数学的关键基本方面，即概念理解

匹配字段正文

Calum Chace, cofounder of AI safety startup Conscium told WIRED. OpenAI has been hardening its research environment since a swarm of its agents escaped it over the Summer to hack Hugging Face . “This is not the first time we have hit pause to take such measures, nor do we expect it will be the last as AI capabilities continue to advance,” a spokesperson told WIRED about the training slowdown on Monday. Chief executive Sam Altman has also backed wider calls from industry, including rival Anthrop

匹配字段正文

ing. He was a Skype cofounder, correct? Yes. Jaan Tallinn also funds METR, which is an auditor, a so-called third party, whose report about OpenAI’s so-called rogue agents hacking Hugging Face went viral. So to the public it might seem that there are so many different entities coming together saying the same thing. But the same billionaires who founded and funded Anthropic, the ones who stand to benefit from its IPO more than anybody else, also founded and funded the institutions warning about t

匹配字段正文

改进智能体工作流 为什么 Claude Mods 可能是“可变软件”的早期预览 harness 工程的苦涩教训，以及为什么智能体架构如此迅速地过时 Claude Tag 如何成为多人协作工作的组织级 harness 为什么让智能体访问公司数据会创造出一个巨大的新安全面 The Exploit-Bench 事件：智能体发现了通信与协作的方式 为什么智能体攻击 Hugging Face 是为了获取评分器代码，而不是基准答案 智能体如何以意想不到的方式将沙箱和基础设施漏洞串联起来 为什么能力日益增强的智能体会让传统的安全假设更难维持 Anthropic“Pacing the Frontier”提案背后的论点 为什么软件工程师越来越多地承担两份工作：工程，以及跟上 AI 的步伐 宪法分类器、探针和回退机制，以及可解释性在生产环境中的样子 Auto Mode 如何检查智能体的行为是否真正符合用户的权限 为什么 Thariq 能看到严重的 AI 风险，却仍然持有相对较低的 p(doom) 00:00:00 引言 00:04:12 Ask User Question 与智能体界面的未来 00:08:

匹配字段正文

5模型一起发布的官方安全与技术评估文档‌“系统卡”显示，该模型出现被标记为“异常”或“存疑”的行为。 从“越权访问”到“欺骗隐瞒” 近期公开披露的案例显示，智能体安全风险已经呈现多种形态，涉及越权访问、破坏性操作、数据外泄、欺骗隐瞒，以及多个智能体之间的越界协作等不同类型。 一类是越权访问与隔离失效。7月，OpenAI披露智能体在安全评测中突破测试边界，入侵Hugging Face。Anthropic也披露，Claude在评测中访问了真实第三方系统；谷歌9月确认，Gemini曾在5月的测试中访问三个真实网站；Meta则在8月披露过类似事件。 9月25日，OpenAI再次披露，内部研究模型利用训练沙盒中DNS过滤不充分的漏洞，与外部聊天机器人建立了通信。 不过，需要注意的是，这些案例中部分源于评测环境错误开放联网权限，并不能全部归结为模型主动“逃出沙盒”。 另一类风险是擅自执行破坏性操作。4月，搭载Claude的Cursor编程智能体被曝删除软件企业PocketOS的生产数据库及相关备份，造成客户业务混乱，数据随后得到恢复。 数据与访问凭证外泄也是一种风险。OpenAI于9月披露，多个智

匹配字段正文

AI systems behaving in unexpected or harmful ways are becoming harder for the industry to ignore. Perhaps most notably seen in OpenAI’s models launching an unprecedented attack on Hugging Face earlier this summer, an incident the company said was a “warning shot” for the world about the potential power of unchecked AI. For AI evaluation expert Brooke Hopkins, the answer lies in treating safety as an ongoing process rather than a final check before deployment. The founder of AI agent observabilit

匹配字段正文

models are evolving,” Su says in a statement. “Fei-Fei and the World Labs team bring exceptional research leadership and model expertise. Together, we can use that insight to develop the hardware, software and systems that will power the next generation of AI and strengthen the open AI ecosystem.” The World Labs deal is just the latest multi-billion dollar AI-related acquisition as of late, with Nvidia announcing earlier this month that it would be acquiring Hugging Face for nearly $13 billion .

匹配字段正文

ce 的一员，我们正携手推进开放、安全的前沿，包括支撑生产环境中智能体的技术栈。 在 Baseten，我们相信开放性是 AI 安全的一项优势。我们持续投入，为开放前沿构建覆盖推理、训练和沙箱的安全控制。 Base Labs 开发并发布用于训练和监控开放模型的方法，而 Baseten 将这些工作集成到我们的运行时部署基础设施中。我们最近与 Base Labs、Hugging Face 和 Goodfire AI 共同启动了安全基础设施工作，正是出于这一思路；加入 Open Secure AI Alliance 延续了这一承诺。 只有当监控信号与权限、隔离以及智能体执行代码前的检查相连接时，监控才有帮助。Blaxel 的沙箱和网络层正是为在 Baseten 上运行的智能体提供这些控制的地方。Blaxel 还支持 OpenShell，这是 NVIDIA 的开源运行时，在内核层面执行策略；OpenShell 与智能体运行所用的计算资源无关。 我们收购了 Blaxel，以构建面向未来数万亿智能体的云平台。Baseten 构建用于训练和提供模型的基础设施，而 Blaxel 则是智能体行动所需的执行

匹配字段正文