NAll news

All news

Friday39 items

  1. Ma Dongxi NLPAnalysis
    AI score58

    Reinforcement Learning Terminology for LLM Readers: How a Single Score Is Distributed to Each Token

    The article uses DeepSeek-R1-Zero's pass@1 on AIME 2024 rising from 15.6% to 71.0% as an example to show that a rule-based judge gives only a single score for the entire response, whereas training needs to adjust the probability of each token. The author uses the sampling example "Is OpenAI open?" to explain in turn policy, state, action, reward, return, value, Q, advantage, baseline, TD error, GAE, clipping, and KL, and points out that PPO uses a critic to learn V, while GRPO uses the average score within a group to replace V.

    #Tutorials / practice#Reasoning#English text is being prepared.

  2. Shao Meng
    AI score58

    poteto shares the Grok Bot template tibot for monitoring X mentions and keywords

    poteto has publicly shared tibot, the Grok Bot template they use to manage their X account. It can monitor mentions and keywords, generate short digests daily or hourly, filter out noise, translate non-English posts, and optionally auto-block harassing accounts. tibot does not post on your behalf; poteto still handwrites all posts and long-form articles. Automatic blocking requires users to connect their own X API app and will consume API credits.

    Quoted postlauren · @poteto

    i've gotten a lot of questions about how i manage my X account with grok bot, so i'm sharing my bot tibot today! he watches your mentions and: • finds every post about you or any keywords, tagged or not • gives one short, sorted digest a day (or hourly, your call) • skips spam and noise • translates non-English posts • can auto block trolls and abuse • never posts for you i still write all my posts and articles by hand but tibot has been very helpful with keeping me in the loop. hire tibot today! https://t.co/OaEAunGL8Y

    Same story · 2 reports

    #Product updates#Agents#English text is being prepared.

  3. Testing CatalogAnalysis
    AI score58

    Artificial Analysis Intelligence Index v5 is live, Claude…

    Artificial Analysis Intelligence Index v5 is live, featuring 2 new benchmarks and 3 upgrades, of which 60% are based on private test sets or held-out answers. Claude Opus 5.5 (max) leads with 58 points, followed closely by GPT-6 Astra (max) at 57 points and GPT-6.1 Sol (max) at 55 points, with Gemini 4 Argon (high) further back at 50 points. The gap between open-weight models and the leader has widened from 3 points to 12 points; among released weights, DeepSeek V4.1 Flash 1017 (max) and GLM-5.3 (max) lead with 32 points.

    Quoted postArtificial Analysis · @ArtificialAnlys

    Announcing Artificial Analysis Intelligence Index v5, featuring 2 new benchmarks and 3 upgrades. For the first time, the majority of the Index is built on private datasets The Artificial Analysis Intelligence Index is our composite measure of model capabilities across tasks reflecting real-world usage today. v5 is our biggest upgrade since v4 launched in January. With v5, 60% of the Index is now based on evaluations with private test sets or held-out answers. Alongside Intelligence Index v5, we’re launching the Artificial Analysis Progress Index v1, which tracks model capability over time on one continuous scale. New to the Intelligence Index: ➤ FrontierCode 2.0: Cognition's newly-launched private benchmark of real-world coding tasks, graded on whether a repository maintainer would merge the agent's change ➤ Terminal-Bench-Science: 70 agentic scientific research tasks, run in a terminal environment Upgraded: ➤ HLE-Diamond (replacing Humanity's Last Exam): a refined 1,000-question version of Humanity's Last Exam, split evenly between reasoning and knowledge questions ➤ AA-Omniscience v1.1: An updated dataset with saturated and public set questions removed, new questions added, and ambiguous or erroneous questions revised ➤ CritPt v2: a private, 40-question hard subset of CritPt v2, revised by the CritPt authors and run with a code-execution tool Staying in the Index: AA-Briefcase, AutomationBench-AA, GDP.pdf and Terminal-Bench 4.0 Removed from the Index: GDPval-AA v2.1, SciCode, and AA-LCR v1.1 Key results: ➤ Claude Opus 5.5 (max) leads at 58, ahead of GPT-6 Astra (max, 57) and GPT-6.1 Sol (max, 55): Anthropic and OpenAI hold the top five models. Gemini 4 Argon (high) is next at 50 ➤ The open weights gap widens from 3 points to 12: Kimi K3.1 Pro (weights not yet released) is the top open weights model at 46. Among released weights, DeepSeek V4.1 Flash 1017 (max) and GLM-5.3 (max) lead at 32 - 26 points behind the leader and below Claude Haiku 5.5 (max, 36) ➤ Cost per task varies by more than 13x among the leading models: of the 10 highest-scoring models shown, GPT-6.1 Terra (max) is the cheapest at $0.96 per Intelligence Index task and Claude Fable 5.1 (max) the most expensive at $13.04

    Same story · 8 reports

    #Evaluations / benchmarks#Anthropic#OpenAI

  4. A Yi AI Notes
    AI score53

    The Trick of Using a Scheduled Ping to Start the Codex 5-Hour Quota Window Early

    The author shares a Codex quota window trick figured out by the community: the 5-hour quota starts counting down from the first message of the day, so at 7:00 AM you can use Grok Bot to automatically run codex exec ping to trigger the countdown; after the quota resets at 12:00 noon, it covers the afternoon work hours. The article provides a ready-to-copy scheduled prompt and notes that this mechanism comes from community testing rather than OpenAI's official documentation, and that read-only parameters must be included when executing.

    #Tutorials / practice#English text is being prepared.#OpenAI

  5. Ayi AI NotesAnalysis
    AI score51

    The author built an "EQ plugin" with Claude Opus 5.5 and used UUMit to turn a local service into a team link

    The author used Cursor together with Claude Opus 5.5 to build a zero-dependency local web tool called "EQ plugin," which rewrites emotionally charged drafts according to the four elements of Nonviolent Communication, and includes seven types of communication targets and three tone levels: gentle, firm, and direct. The tool runs on the local machine at 127.0.0.1:8787, and the UUMit client generates a fixed HTTPS link to distribute to colleagues, with no need to buy a server or file for registration, and permissions can be granted and revoked per person. The article also provides an 800-character limit per input, default rate limits of 10 times per minute and 200 times per day, as well as the source code and System Prompt for two GitHub repositories.

    #Tutorials / practice#Agents#English text is being prepared.

  6. Testing CatalogAnalysis
    AI score70

    Gemini 4 Carbon Tops Text Arena, Ranks Second in Agent Arena

    Gemini 4 Carbon (High) took first place in Text Arena with 1528 points, surpassing Gemini 4 Argon (High)'s 1520 points. It ranks second in Agent Arena, behind only Claude Opus 5.5, and has entered a new Pareto frontier; in Web Dev Arena it ranks fourth with 1768 points, two points behind third-place Claude Sonnet 5.5. Arena data states its median cost per Agent Arena task is $1.80, 72% lower than top-ranked Claude Opus 5.5 (Max). Its Artificial Analysis V5 score is 59.

    Quoted postArena.ai · @arena

    Exciting news: Gemini 4 Carbon (High) from @GoogleDeepMind just made a huge debut on Arena, landing #1 in Text Arena, #2 in Agent Arena, and #4 in Code Arena: WebDev. Back at the top, Gemini 4 Carbon delivers frontier intelligence and reshapes the Pareto frontier across three arenas. In Agent Arena, Gemini 4 Carbon ranks #2 with a +16.2% net improvement across +21.9K real-world agentic sessions from our global community, surpassing top models GPT-6 Astra and Fable-5.1. At $1.80 median cost per task, Gemini 4 Carbon (High) costs 72% less than the #1 ranked Claude Opus 5.5 (Max). Its standouts by key signals: - #1 Steerability (+18.6%): can the model course-correct when you push back? - #2 Praise vs Complaint (+37.9%): implicit sentiment in users' reactions - #3 Confirmed Success (+18.7%): explicit “yes, that worked” from users In Code Arena: WebDev, Carbon scored 1768 pts landing #4: - Just 2 pts below #3 Claude Sonnet 5.5 (xHigh), at 1770 pts - 89 pts above #11 Gemini 4 Argon (High), at 1679 pts By category, it ranks #3 in Gaming and Simulations, and #5 in Brand & Marketing. In Text Arena, Gemini 4 Carbon (High) is the new #1 with 1528 pts, now ahead of Gemini 4 Argon (High) at #2 with 1520 pts. Congrats to the @GoogleDeepMind team on this huge milestone!

    Same story · 2 reports

    #Evaluations / benchmarks#Google#Agents

  7. Shao MengAnalysis
    AI score59

    DeepLearning.AI Launches Context Engineering Course, Teaching Agent Context Management

    DeepLearning.AI has launched a new Context Engineering course, taught by Chroma CEO Jeff Huber and CMPND CEO Drew Breunig, framing context engineering as the main thread for keeping Agents reliable and token-efficient in complex multi-step tasks. The course builds a harness from scratch in Python and uses traces and evals to quantify each change, covering three capabilities—Agentic Search, Context Management, and Agentic Memory—across four modules, with labs mostly consisting of about 1-hour coding exercises. The course assumes intermediate Python, familiarity with LLM applications and tool calling, and basic retrieval.

    #Tutorials / practice#Agents#RAG

  8. PromptArmorFirst-partyAnalysis
    AI score60

    PromptArmor testing found that DeepSeek, Kimi, GLM, and Qwen are 88% more susceptible to cyberattacks than…

    PromptArmor tested its most widely circulated attack samples across multiple models and found that DeepSeek, Kimi, GLM, and Qwen models are 88% more susceptible to cyberattacks than GPT, Claude, and Gemini models. The tests covered multiple cyber risk dimensions, including susceptibility to phishing, data exfiltration, and running malware, and the authors described the differences as significant.

    #Safety / alignment#Evaluations / benchmarks#DeepSeek

  9. PromptArmorFirst-partyAnalysis
    AI score52

    PromptArmor tested multiple models for cyberattack risk, claiming DeepSeek, Kimi, GLM, and Qwen are more...

    PromptArmor said it used multiple highly viral attack samples to test cyber risks including phishing, data exfiltration, and propensity to run malware, and that DeepSeek, Kimi, GLM, and Qwen models were 88% more likely to be attacked than GPT, Claude, and Gemini. This conclusion comes from its main tweet; the specific testing methodology and sample size were not explained in the tweet.

    #Safety / alignment#Evaluations / benchmarks#DeepSeek

  10. Shao MengAnalysis
    AI score71

    Addy on how to use Claude Code in the Claude Opus 5.5 era

    Addy Osmani discusses how to use Claude Code in the Claude Opus 5.5 era, arguing that starting with the Claude 5 series, users should reduce piling up prompts and hand-holding task breakdowns, and instead use a single prompt that clearly states the goal and acceptance criteria, letting the model run to completion on its own and then stop. He recommends starting from /effort medium; when the goal is unclear, first have Claude interview you to produce a spec, then open a new session to implement it. For Claude Opus 5 and Claude Fable 5, he removed more than 80% of the Claude Code system prompt, with no noticeable drop in coding benchmarks. He also mentioned that the Claude Code interactive terminal recently defaulted to auto mode, and said this is not a safety guarantee; sub-agents should be split less by hand, and a single agent plus skills often yields better quality.

    Quoted postAddy Osmani · @addyosmani

    https://t.co/1txsLAxxM5

    Same story · 2 reports

    #English text is being prepared.#Agents#English text is being prepared.

  11. Shao MengAnalysis
    AI score58

    Matt Palmer demonstrates using Cursor Cloud Agent to generate a UI motion video against real Slack

    Matt Palmer demonstrated opening Chrome and real Slack on the Linux remote desktop of a Cursor Cloud Agent, having the Agent generate a Slack UI motion video with code, and then iterating and revising after checking easing and motion patterns against the real interface. He broke the process down into an inner loop (the Agent does the work, checks against the real thing, and fixes) and an outer loop (a human provides references, preferences, and permissions and accepts the output), and proposed the Do→Codify→Generalize ladder, packaging one success into a Skill and then generalizing it. He emphasized that Slack was not decompiled; he merely gave the Agent a reference and asked it to extract visual patterns by comparison.

    Quoted postmatt palmer · @mattyp

    Agents rebuilt Slack as video (using code) by looking at the real thing 0:00 Why 0:25 @cursor_ai agents & computer use 1:03 Video as code 1:56 Slack 2:54 A chat, an agent, a computer 3:53 Inner loop and outer loop 5:04 The ladder 7:47 Skills 8:14 Feedback loops 10:02 Recap https://t.co/S91yI9vLop

    #Tutorials / practice#Agents#English text is being prepared.

  12. Rohan PaulAnalysis
    AI score71

    Gemini 4 Carbon Tops the Artificial Analysis Intelligence Index with 59 Points

    Google's Gemini 4 Carbon ranks first on the Artificial Analysis Intelligence Index with 59 points, 1 point higher than Claude Opus 5.5's 58 points and 2 points higher than GPT-6 Astra's 57 points. Artificial Analysis found that Carbon won 4 of 9 benchmarks (HLE-Diamond, Terminal-Bench-Science, AA-Omniscience, AutomationBench-AA), but it does not lead in coding and knowledge work. Calculated at standard prices, it costs $7.76 per index task, about 23% lower than Opus 5.5's $10.04, but higher than GPT-6 Astra's $6.25.

    Quoted postDemis Hassabis · @demishassabis

    No.1 💎 https://t.co/k4Werb6ZGV

    #Evaluations / benchmarks#Google#Anthropic

  13. Artificial AnalysisAnalysis
    AI score71

    Artificial Analysis tests GitHub Copilot Fusion: $5.59 cost per task…

    In Coding Agent Index v2.0, Artificial Analysis benchmarked GitHub Copilot's Copilot Fusion, which costs $5.59 per task, compared with $11.42 for Claude Code with Opus 5.5 (max), while performing only 2 points lower. Copilot Fusion is a runtime orchestration system that selects and coordinates among Anthropic and OpenAI models such as Claude Opus 5.5, Claude Sonnet 5.5, GPT-6 Astra, and GPT-6 Sol. It leads on Terminal-Bench 4.0 with a score of 71, ranks third on FrontierCode with 51 points, and uses 7.2M tokens per task, about half that of Opus 5.5 in Claude Code.

    Same story · 6 reports

    #Product updates#Agents#English text is being prepared.

  14. Artificial AnalysisAnalysis
    AI score59

    Artificial Analysis releases the image-to-video benchmark AA-Video-I2V v2.1

    Artificial Analysis launches the image-to-video evaluation benchmarks AA-Video-I2V v2.1 and AA-Video-I2V-Silent v2.1, ranking models at up to 1080p resolution across 10 use-case categories, 10 capabilities, and multiple visual styles. The two versions are based on more than 58,000 human preference votes across over 1,000 prompts and more than 44,000 votes across 500 prompts, respectively, with only 5% of prompts made public in the public arena and rotated monthly. In the first version of the leaderboard, Dreamina Seedance 2.5 ranks first overall, statistically tied with the Seedance 2.0 and MiniMax H3 series, at a price of $34.12/min; the open-weight MiniMax H3 (768p) ranks third, at a price of $4.80/min.

    #Evaluations / benchmarks#Video#Image generation

  15. Artificial AnalysisAnalysis
    AI score80

    Gemini 4 Carbon Tops the Artificial Analysis Intelligence Index…

    Artificial Analysis evaluations show that Google's Gemini 4 Carbon tops Intelligence Index v5 with 59 points, ranking first in four of the 9 evaluations: HLE-Diamond, Terminal-Bench-Science, AA-Omniscience, and AutomationBench-AA. On the Coding Agent Index, Gemini 4 Carbon (high) paired with Antigravity CLI scores 59 points, 3 points behind Claude Code paired with Claude Opus 5.5 (max, 62 points). The model is currently available only to some users and has not yet been publicly released; it is priced at $4/$20 per 1M input/output tokens with a 50% discount, has a 1M-token context window, and supports text, image, video, and voice input.

    Same story · 34 reports

    #Evaluations / benchmarks#Reasoning#English text is being prepared.

  16. ArenaAnalysis
    AI score70

    Gemini 4 Carbon ranks 2nd on Agent Arena with a net improvement of 16.2%

    Arena.ai says Gemini 4 Carbon ranks 2nd in the real-task-based Agent Arena with a net improvement of 16.2%, and sits on the Pareto frontier for cost, time, and output tokens per task. Compared with Gemini 4 Argon, which ranks 7th with an 11% improvement, Carbon is 5 places higher and 5.2 percentage points better; the gains mainly come from a +13.2 percentage point increase in the positive-feedback-to-complaint ratio, +5.6 percentage points in confirmed success, and +4.8 percentage points in steerability. Arena says Carbon's availability will be announced by Google DeepMind.

    Quoted postArena.ai · @arena

    Exciting news: Gemini 4 Carbon (High) from @GoogleDeepMind just made a huge debut on Arena, landing #1 in Text Arena, #2 in Agent Arena, and #4 in Code Arena: WebDev. Back at the top, Gemini 4 Carbon delivers frontier intelligence and reshapes the Pareto frontier across three arenas. In Agent Arena, Gemini 4 Carbon ranks #2 with a +16.2% net improvement across +21.9K real-world agentic sessions from our global community, surpassing top models GPT-6 Astra and Fable-5.1. At $1.80 median cost per task, Gemini 4 Carbon (High) costs 72% less than the #1 ranked Claude Opus 5.5 (Max). Its standouts by key signals: - #1 Steerability (+18.6%): can the model course-correct when you push back? - #2 Praise vs Complaint (+37.9%): implicit sentiment in users' reactions - #3 Confirmed Success (+18.7%): explicit “yes, that worked” from users In Code Arena: WebDev, Carbon scored 1768 pts landing #4: - Just 2 pts below #3 Claude Sonnet 5.5 (xHigh), at 1770 pts - 89 pts above #11 Gemini 4 Argon (High), at 1679 pts By category, it ranks #3 in Gaming and Simulations, and #5 in Brand & Marketing. In Text Arena, Gemini 4 Carbon (High) is the new #1 with 1528 pts, now ahead of Gemini 4 Argon (High) at #2 with 1520 pts. Congrats to the @GoogleDeepMind team on this huge milestone!

    Same story · 2 reports

    #Evaluations / benchmarks#Agents#Google

  17. ArenaAnalysis
    AI score65

    GPT-6.1 Terra ranks 8th in Agent Arena, with a median cost of $0.62 per task

    Arena's empirical results, based on nearly 9,000 agent sessions, show that OpenAI's GPT-6.1 Terra (Max) ranks 8th in Agent Arena, with a net improvement of +10.6% and a median cost of $0.62 per task. This cost is 43% lower than the $1.09 of 7th-place Gemini 4 Argon (High), 30% lower than the $0.89 of 6th-place GPT-6.1 Sol (Max), and 88% lower than the $5.16 of 5th-place GPT-6 Astra (Max). In the subcategories, it ranks 3rd in Bash Reliability (+13.2%), 7th in Steerability (+13.3%), and 9th in Praise vs Complaint (+16.8%).

    Quoted postArena.ai · @arena

    Real-world results are in: GPT-6.1 Terra (Max) by @OpenAI just landed #14 in the Code Arena: WebDev with 1671 pts, delivering Pareto level results! That’s +151 pts over GPT-5.6 Terra, now at #56. At a median cost of $0.40 per task, GPT-6.1 Terra costs 28% less than GPT-5.6 Terra ($0.56 per task), reshaping the Pareto frontier. Compared to other models in the family, GPT-6.1 Terra (Max) is incredibly efficient: - 81 pts below #4 GPT-6.1 Sol (1752 pts), at 75% lower median cost per task. - Just 6 pts below #11 GPT-6 Sol (1677 pts), at 76% lower median cost per task Congrats to the @OpenAI team on this release!

    #Evaluations / benchmarks#Agents#OpenAI

  18. ArenaAnalysis
    AI score60

    Arena Analysis: Agent Arena Cost per Task: GPT-6.1 Sol at $1.91

    Arena's statistics on Agent Arena show that OpenAI's GPT-6.1 Sol (Max) costs $1.91 per task, 85% less than Claude Opus 5.5 (Max) at $12.51 and 87% less than GPT-6 Astra (Max) at $14.93. Arena notes that the sources of these two cost advantages differ: relative to Astra, Sol's advantage comes mainly from a lower token unit price; relative to Opus, it comes mainly from consuming fewer tokens per task. Cost per task is calculated by multiplying the number of input, output, and cache-read tokens by their respective unit prices, and Arena pinpoints the causes of cost reductions by comparing prices and token usage.

    #Evaluations / benchmarks#Agents#OpenAI

  19. Artificial AnalysisAnalysis
    AI score60

    Fish Audio Drama 3 ranks first among open-weight models in two speech arenas, and first in pronunciation robustness

    Artificial Analysis evaluations show that Drama 3, released by Fish Audio, has an Elo of 1,278 in the Provider Voice scenario of Text to Speech (1,512 arena appearances), ranking 6th overall and 1st among open-weight models; in the Controlled Voice scenario, its Elo is 1,085 (1,958 appearances), ranking 12th overall and 1st among open-weight models. On pronunciation robustness, Drama 3 ranks 1st with 94.6% (2,157 rated clips), higher than the 93.2% of Ming-3.0-tts and Drama 3 Preview. Its generation speed is 66.6 characters per second, its pricing is $15/1M characters, and its weights are open under the Fish Audio Community License.

    #Evaluations / benchmarks#Speech#Open-source ecosystem

  20. Artificial AnalysisAnalysis
    AI score62

    Phonic 1 Tops the Artificial Analysis Speech-to-Speech Index, Leading Grok with 87.1 Points…

    Artificial Analysis evaluations show that Phonic's Phonic 1 (High) ranks first among 22 models on the Speech to Speech Index with 87.1 points, leading Grok Voice 3.0's 86.9 points by a narrow margin. The model ranks first on the Tau Voice agent task with 79.9%, leading the second place by 5.4 percentage points; it ranks third on Big Bench Audio with 98.9%, and its Time to First Audio is 0.61 seconds. Phonic 1 is available through the Phonic API, with input audio costing $0.07 per minute and $4.20 per hour.

    Same story · 2 reports

    #Evaluations / benchmarks#Speech#Agents