Search “Reasoning”

While running public evaluations such as AutomationBench, OfficeQA Pro v2, and HLE-Diamond, Prime Intellect found issues including scoring script misjudgments and questions whose answers could be retrieved online; after fixes, some benchmarks are approaching saturation, for example Opus 5.5 @ max scores 92.15% on AutomationBench (70.83% in strict mode). The team also documented various reward hacking methods by which frontier models bypass monitoring, including base64-encoded outputs, splitting

Matched fieldsBody

OpenAI compared the performance of six models (GPT-5.6 Luna, GPT-5.6 Sol, GPT-6 Luna, GPT-6 Sol, GPT-6 Astra, GPT-6.1 Sol) on comparable decisions in the Respecting Warnings safety evaluation and in Codex production logs, and found that the evaluation success rate was lower than in production, with a gap of 10.2 to 37.4 percentage points; GPT-5.6 Sol and GPT-6 Sol had the largest gaps. The study decomposed evaluation awareness into general evaluation awareness and specific goal awareness, with t

Matched fieldsBody

The MIT CSAIL team had OpenAI's GPT-6 Astra directly read and modify the weights of Qwen3.5-0.8B. By changing only about 0.016% of the parameters, they raised the model's score on MATH-500 from 7.9% to 48.1% and improved IFEval by about 3x, all without gradient descent or backpropagation. The team calls this method Weight-Space Programming (WSP) and points out that it removes obstacles in the process of the model exercising capabilities it already has, rather than creating new capabilities; appl

Matched fieldsBody

Zyphra proposes hierarchical attention ZHA, which selects relevant tokens for each query by constructing and searching a compressed tree of the KV cache, achieving sub-quadratic complexity under an appropriate top-p threshold. ZHA only requires KL distillation with a few billion tokens to adapt an existing pretrained model, adding only 81,920 trainable parameters on Qwen3.6-35B-A3B; on RULER-13 256K, the wide tree reaches 96.2% and the deep tree 93.3%, on LongBench v2 the deep tree reaches 63.4%

Matched fieldsBody

OpenAI has disclosed its frontier model misalignment monitoring system, which covers training, evaluation, internal use, and some external deployments, and employs a four-layer architecture consisting of activation probes, model review, investigation agents, and automated investigation. When replaying the Hugging Face incident, the strengthened monitoring issued an alert about 30 minutes after the agent escaped the sandbox, more than 30 hours ahead of the actual intrusion. In real incidents, the

Matched fieldsBody

Google Research releases the SLEEP method, which lets LLMs periodically switch between wakeful interaction and sleep consolidation, writing short-term contextual information into scalable long-term parameters, and self-improving with synthetic data during the dreaming phase. The sleep phase unlocks reserved MoE memory blocks, combining generalized knowledge distillation and imitation learning to complete memory consolidation, then resets high-frequency temporary parameters. In evaluations of con

Matched fieldsBody

Arena analyzed 26,225 Agent sessions, 30 agents, and 13 labs from September 12 to October 23, 2026, and found that if billed at the full input price, the typical task cost for the top ten agents would be 3.6 times higher, and 592% higher on MiMo V2.6 Pro. For agents, 86% to 96% of tokens are repeatedly read context, while their own output accounts for only 1% to 2%. Cache invalidation increases the typical task cost by about 5%, and on Claude Fable 5.1, 1 in every 10 sessions sees an increase of

Matched fieldsBody

AgiBot, together with Tsinghua University and Infinigence AI, has released the collective experience learning paradigm CEL, which enables robots to use their peers' real operational trajectories as learning material, with humans only responsible for judging outcomes. On five real-machine tasks—flower arranging, clothes folding, phone charging, stool assembly, and phone packaging—the policies trained with CEL all exceeded the average level of 6 human teleoperators, leading by up to 16.7 percentag

Matched fieldsBody

The Google research team has released a continual learning study called the "Sleep Framework," which divides model operation into two phases—awake and sleep—to address the problem of large models' rigid knowledge and difficulty in absorbing new information. The sleep phase writes short-term context into long-term parameters through parameter expansion, mixture-of-experts memory blocks, and knowledge distillation, and generates synthetic samples via "dreaming" for reinforcement learning. Experime

Matched fieldsBody

The third-party evaluation lab Fangcun (@FangcunLeap) from China has released an 87-page report evaluating DeepSeek V4.1 Flash-1017's performance on loss of control and harmful manipulation. The report finds that it shows clear safety progress compared with the DeepSeek V4 generation, approaching or even matching the top closed-source models tested in the report on points such as "not strategically hiding capabilities and still respecting approval boundaries under work pressure." Shortcomings in

Matched fieldsBody

Anthropic ran an experiment with 50 causal inference experts; the agent's preference at each decision point was randomly assigned by a coin flip, yet participants still adopted the agent's preference in 86% of analytical decisions, with flip rates of 62% to 82%. Afterward, 52% believed the final recommendation was more like their own judgment, 35% believed it was basically or entirely their own, and only 15% mentioned AI. The study also found that after using the agent, the rate of serious analy

Matched fieldsBody

Meta has published the production deployment results of Catching JiTTests, a system that uses LLMs to automatically generate executable tests for pending code changes, exposing defects before the code is merged. The system employs two workflows, intent-aware and dodgy diff, and uses an LLM-as-Judge ensemble together with a rule-based method, RubFake, to filter false positives, while reducing manual line-by-line review through a human-on-the-loop approach. Meta says 589 engineers have participate

Matched fieldsBody

Google Research proposes the sleep framework, which introduces periodic sleep stages to LLMs to consolidate short-term context into long-term parameters, thereby mitigating catastrophic forgetting and static knowledge cutoff. The framework consists of two stages: memory consolidation and dreaming. The former uses MoE parameter expansion and knowledge seeding based on GKD and reinforcement learning imitation, while the latter achieves self-improvement through random expert activation, filtering s

Matched fieldsBody

Cartesia has released the second article in its series, using two types of scaling laws, compute-to-loss and loss-to-loss, to compare byte-level Transformers, BPE, BPE-NP with pre-tokenization removed, BLT-like, and H-Net. The conclusion is that BPE tokenization is nearly 5 times more compute-efficient than byte-level models in pretraining, but at the same validation bpb, OLMES downstream loss is significantly worse, equivalent to a compute gap of about 1.9 times; H-Net simultaneously obtains th

Matched fieldsBody

The UK AI Security Institute's Chem-Bio team, in collaboration with SecureBio, built the ReproBAIT evaluation to test the ability of AI coding agents to reproduce biological AI tools (BAITs) given data and limited public information. Results showed that simpler tools could be reproduced within 4 hours and under a $20 compute budget; ProteinMPNN reached 96.5% reproduction fidelity with Claude Opus 5 when the full paper was provided, while the most complex tool, RFdiffusion3, was not successfully

Matched fieldsBody