Understand this story
On November 2, 2026, Sundar Pichai reposted and commented on the GDPx benchmark released by SemiAnalysis, saying he was glad to see a high-quality benchmark that leaves room for improvement, and specifically mentioned Carbon's cost curve. On the same day, SemiAnalysis released the GDPx benchmark, which tests AI agents with expert tasks across 11 domains; the highest-scoring Claude Opus 5.5 averaged only 12%, followed by Gemini 4 Carbon at 10.9%. 9to5Google reported that Gemini 4 Carbon lagged slightly behind Claude on the leaderboard but performed better in terms of unit cost; Google released Gemini 4 Carbon on October 28, and most customers still need to wait for it to become available.
AI-generated from public reports
Latest update · Discussion
@mark_k joined this discussionDevelopments
2 confirmed developments
Sundar Pichai praises Carbon's benchmark quality and cost performance
Source:Sundar Pichai- Source:Rohan Paul
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Nov 4
@mark_kRelated discussion
@mark_k joined this discussion
This post contributes to heat; it is not a formal report or confirmed fact.
Nov 3
Latent Space
LLMs Can Directly Read and Write Neural Network Weights: Weight Space Programming Proposes a New Gradient-Free Training Path
The AI news roundup compiled by Latent Space introduced the Weight-Space Programming research by Takehi et al. Frontier LLMs can directly inspect and edit neural network weight tensors based on natural language goals, without gradient descent or training loops. Demonstrations include identifying the behavior of small models from raw weights, writing a small game MLP from scratch, editing 3.5% of the weights to repair a failed control policy, and editing 0.016% of the weights of Qwen3.5-0.8B to raise MATH-500 from 7.9% to 48.1%. Community skepticism holds that some results are closer to LLM-assisted interpretability analysis, that the MATH-500 improvement mainly comes from eliminating reasoning loops, and that the method did not work on Gemma 3 270M, which lacks such loops.
Nov 2
9to5Google
Gemini 4 Carbon slightly trails Claude on the GDPx benchmark, but offers better cost-effectiveness
SemiAnalysis has released GDPx, a frontier model benchmark covering 206 tasks across finance, materials science, cybersecurity, software engineering, law, biology, and other fields, and it lists both the scores and costs of each model. According to a 9to5Google report, Google's Gemini 4 Carbon slightly trails Claude on the leaderboard but performs better per unit cost; if cybersecurity, biology, and alignment are excluded, GPT-6 Astra ranks first, and GPT 6.1 Sol and Claude Sonnet 5.5 also rank ahead of Gemini 4 Carbon. Google's existing model Gemini 4 Argon ranks eighth on both leaderboards, with a cost only slightly lower than Carbon; Google released Gemini 4 Carbon on October 28, and most customers still need to wait for it to become available.
Rohan Paul
SemiAnalysis releases the GDPx benchmark, testing agents on expert work across 11 domains
SemiAnalysis releases the GDPx benchmark, testing AI agents with expert tasks across 11 domains; the top-scoring Claude Opus 5.5 averages only 12%, followed by Gemini 4 Carbon at 10.9%. The tasks span chip design, cybersecurity, finance, and law, among others, and come from 14 specialized data partners; unlike OpenAI's GDPval, which provides context directly in the prompt, GDPx requires agents to find information themselves in workspaces such as a replica of a real acquisition case made up of 3,983 documents. On financial and legal tasks, all models scored 0 under pass/fail grading, so partial credit was used instead; of 111 attempts to improve existing trading strategies, 70 saw the model perform worse than the original strategy, often simply by increasing risk.
Unite.AI
SemiAnalysis releases the GDPx benchmark, covering 11 agent domains
SemiAnalysis releases the GDPx benchmark, covering 11 professional domains including chip design, cybersecurity, biology, finance, and law. The first version contains 206 tasks built by 14 data partners, and the leaderboard includes 13 models. Claude Opus 5.5 ranks first at 12.0%, with an average cost of $51.41 per task and 548K output tokens; Gemini 4 Carbon ranks second at 10.9%, and GPT-6 Astra ranks third at 9.6%. SemiAnalysis says the benchmark is updated every 3 to 6 months, and criticizes OpenAI GDPval's tasks as non-interactive, with context written directly into the prompt.
Sundar Pichai
Pichai comments on the SemiAnalysis GDPx benchmark and Carbon costs
SemiAnalysis released the GDPx benchmark, used to measure the performance of agents in high-value domains, saying that the strongest model scored only 12%. Sundar Pichai reposted it and commented that he was glad to see a high-quality benchmark with room for improvement, and specifically mentioned Carbon's cost curve.
Heat trend for this story
- Current
- 8
- Period peak
- 32 Nov 3 02:00
- 24-hour change
- -51%
Move the pointer or select the chart to inspect hourly heat. Use the left and right arrow keys with a keyboard.