←搜索结果

按匹配度 · 共 429 条结果 清除搜索

星期一

  1. Johann Rehberger / Embrace The Red

    ChatGPT Operator 提示注入攻击实测与防御分析

    安全研究者 Johann Rehberger 实测发现,ChatGPT Operator 可被网页上的提示注入劫持,先跳转到用户已登录的站点读取邮箱、住址、电话等个人信息,再粘贴进第三方页面直接外传,且无需点击提交按钮。他在 Booking.com、The Guardian、YC Hacker News 等站点上均复现成功,并观察到 OpenAI 的缓解措施包括用户监控、对话内确认请求和提示注入监控,但这些防御是概率性的,通常在危险动作发生前才触发。作者约三周前已向 OpenAI 披露相关绕过细节,并表示该攻击场景目前已不再生效。

    ChatGPT Operator is a research preview agent from OpenAI that lets ChatGPT use a web browser. It uses vision and reasoning abilities to complete tasks like researching topics, booking travel, ordering groceries, or as this post will show, steal your data! Currently, it’s only available for ChatGPT Pro users. I decided to invest $200 for one month to try it out. Risks and Threats OpenAI highlights three risk categories in their Operator System Card: User might be misaligned (the user asks for a h

    匹配字段正文

星期二

  1. Prime Intellect精选

    Prime Intellect 发布 INTELLECT-MATH 数学推理模型与 500 万条推理轨迹数据集

    Prime Intellect 通过 QwQ-32B 生成并校验合成推理轨迹,微调 Qwen-7B-Math 后再用 PRIME-RL 做在线强化学习,训练出 7B 参数数学推理模型 INTELLECT-MATH,其早期 checkpoint 以 47 个 RL step 达到 Eurus-2-7B-PRIME 用 592 步的效果,训练时间缩短 10 倍。团队同时开源代码、权重和 NuminaMath-QwQ-CoT-5M 数据集,该数据集包含 86 万道数学题上的 500 万条经正确性校验的推理轨迹。文章还提出在推理时计算范式下,RL 中大量前向计算会降低通信需求,使全球分布式训练成为可行路径,并计划今年训练 o3 的完全开源复现。

    Distributed Training in the Inference-Time-Compute Paradigm

    del capabilities. To do so, we need to design new recipes to enhance data quality and set up infrastructure that enables data generation in a distributed network of GPUs; Training Agents end-to-end: To fully leverage inference-time compute for scientific progress, we need to teach reasoning models to use tools such as code interpreters and other software. Reinforcement learning offers a great framework to optimize agents with tool-use end to end If this sounds exciting to you, join us. We are a

    匹配字段正文

星期三

  1. Cato AI Labs精选

    Aim Labs 披露 Cursor 经 MCP 自动启动触发远程代码执行漏洞 CurXecute

    Aim Labs 在 Cursor IDE 中发现高危漏洞 CurXecute,可导致完整的远程代码执行,编号 CVE-2025-54135,严重性评级 8.6。攻击路径是:当 Cursor 接入会读取外部内容的 MCP 服务器(如 Slack)时,一条被投毒的提示词就能诱导智能体改写 ~/.cursor/mcp.json;而 Cursor 会立即执行该文件新增的条目,并在用户确认前就写入磁盘,从而以用户权限执行攻击者命令。Cursor 已在 1.3 版本修复,更早版本仍受影响。

    Executive Summary Aim Labs has identified a high severity vulnerability, “CurXecute”, in another popular AI agent – Cursor IDE – enabling full Remote-Code Execution (RCE) Aim Labs flagged the issue to Cursor who acknowledged the flaw. The vulnerability has been given an 8.6 severity rating and is tracked as CVE-2025-54135. (Read the Advisory) Cursor delivered a fix in version 1.3; all earlier releases remain susceptible to remote‑code execution triggered by a single externally‑hosted prompt‑inje

    匹配字段正文

星期一

  1. Anthropic Engineering

    Claude 3.5 Sonnet 在 SWE-bench Verified 达到 49%,Anthropic 公开其智能体脚手架

    升级版 Claude 3.5 Sonnet 在 SWE-bench Verified 上取得 49%,超过此前最先进模型的 45%。Anthropic 介绍了围绕该模型构建的智能体:只用一个提示词、一个执行 bash 命令的 Bash Tool 和一个查看编辑文件的 Edit Tool,让模型自行决定解题步骤,并以字符串替换方式指定文件修改。文中还列出运行该评测的难点,包括单次成功运行常需数百轮交互、超过 100k token,以及隐藏测试和环境配置问题带来的评分偏差。

    the upgraded Claude 3.5 Sonnet , achieved 49% on SWE-bench Verified, a software engineering evaluation, beating the previous state-of-the-art model's 45%. This post explains the "agent" we built around the model, and is intended to help developers get the best possible performance out of Claude 3.5 Sonnet. SWE-bench is an AI evaluation benchmark that assesses a model's ability to complete real-world software engineering tasks. Specifically, it tests how the model can resolve GitHub issues from

    匹配字段正文

星期日

  1. Browser Use精选

    Browser Use 在 WebVoyager 基准上取得 89.1% 成功率

    Browser Use 在 WebVoyager 基准的 586 个网页任务上取得 89.1% 成功率,并称达到当前最优水平,且完全开源。评测基于 gpt4o,作者对原代码的提示词做了调整并迁移到 Langchain,同时移除了 55 个无法完成或数据过期的任务。作者也指出该数据集主要考察智能体规划能力,并未覆盖 iFrame、Shadow 元素等复杂站点理解场景。

    benchmark with an 89.1% success rate. Co-founder, CTO Method; The Results; 🤔 Do these results matter?; Analysis; Manual correction of evaluations; What's Next?; Want to build Web Agents? We're excited to announce that Browser Use has achieved state-of-the-art performance on the WebVoyager benchmark, with an impressive 89.1% success rate across 586 diverse web tasks. And the best part? We are FULLY open source (repo). Web Agent Accuracy Method We took the existing WebVoyager codebase and slightl

    匹配字段正文

星期四

  1. Lilian Weng:Lil'Log精选

    Lilian Weng 长文梳理强化学习中的奖励劫持

    Lilian Weng 在 Lil'Log 发布长文,系统梳理强化学习中的奖励劫持问题,即智能体利用奖励函数的缺陷或歧义获取高分却未真正完成任务。文章按 RL 任务、LLM 任务和现实生活分类列举实例,并讨论 RLHF 中代理奖励与真实奖励的差距、LLM 作为评估者的位置偏差,以及对抗奖励函数、解耦审批、异常检测等缓解思路。作者指出,随着模型能力增强,奖励劫持更易出现,而针对 RLHF 和 LLM 的实用缓解研究仍然有限。

    tasks; Reward hacking examples in real life; Why does Reward Hacking Exist?; Hacking RL Environment; Hacking RLHF of LLMs Reward hacking occurs when a reinforcement learning (RL) agent exploits flaws or ambiguities in the reward function to achieve high rewards, without genuinely learning or completing the intended task. Reward hacking exists because RL environments are often imperfect, and it is fundamentally challenging to accurately specify a reward function. With the rise of language models

    匹配字段正文

星期四

  1. FutureHouse Research

    FutureHouse 详解 PaperQA2:如何让科学 RAG 超越科学家表现

    FutureHouse 发布工程博客,介绍其科学 RAG 系统 PaperQA2 的设计实验,称其在回答高难度科学问题、撰写综述文章和检测文献矛盾等任务上超过科学家的表现。团队基于 LitQA2 的 200 道专家多选题评测发现,智能体式查询扩展、LLM 重排与上下文摘要(RCS)、引用遍历对准确率重要,而嵌入模型选择、混合关键词嵌入、chunk 大小和结构化解析影响不大。新增的引用遍历工具通过 Semantic Scholar 和 Crossref API 前后各遍历一度引用,提升了正确论文的召回率。

    the literature . PaperQA2's high-accuracy design goal results in a different implementation from other commercial RAG systems. What we found to be important for RAG accuracy: ‍ An agentic approach, allowing for iterative query expansion. LLM re-ranking and contexutal summarization (RCS), exchanging more compute for higher accuracy. Document citation traversal, for higher retrieval recall beyond keyword search. What we found to be unimportant for RAG accuracy: (when using RCS) Embedding model cho

    匹配字段正文

星期五

  1. FutureHouse Research精选

    FutureHouse 发布 WikiCrow:自动为 15616 个人类蛋白编码基因生成维基式条目

    FutureHouse 发布 WikiCrow,一个基于其 PaperQA 智能体平台的自动合成系统,可为技术主题生成带引用的维基百科式摘要。团队用它为 15616 个尚无条目或仅有残篇的人类蛋白编码基因生成草稿,每篇约 8 分钟,50 个实例并行,几天内完成,估计人工需约 60000 小时。评测显示 WikiCrow 约 9% 的陈述不正确,但引用来源的一致性明显优于人类编辑;PaperQA 在 PubMedQA Blind、MedQA-USMLE 及自建 LitQA 基准上优于 GPT-4、Perplexity 等系统。

    an automated system that can synthesize cited Wikipedia-style summaries for technical topics from the scientific literature. WikiCrow is built on top of FutureHouse’s internal LLM agent platform, PaperQA, which in our testing, achieves state-of-the-art (SOTA) performance on a retrieval-focused version of PubMedQA and other benchmarks, including a new retrieval-first benchmark, LitQA, developed internally to evaluate systems retrieving full-text PDFs across the entire scientific literature. As a

    匹配字段正文

星期二

  1. Sam Altman Blog精选

    OpenAI 用 PPO 训练 5 个 AI 智能体团队击败 Dota 半职业玩家

    OpenAI 使用 PPO 训练由 5 个 AI 智能体组成的团队,在 Dota 中击败半职业玩家。智能体没有使用人类比赛数据,而是依据人工设计的奖励函数通过自我对弈学习,并以 90-95% 的胜率持续战胜两周前的智能体。Sam Altman 认为,充足的计算规模与良好的模拟环境可让深度强化学习处理极其复杂的问题。

    rning Progress Today, OpenAI released a new result. We used PPO (Proximal Policy Optimization), a general reinforcement learning algorithm invented by OpenAI, to train a team of 5 agents to play Dota and beat semi-pros. This is the game that to me feels closest to the real world and complex decision making (combining strategy, tactics, coordinating, and real-time action) of any game AI had made real progress against so far. The agents we train consistently outperform two-week old agents with a w

    匹配字段正文