Search “Reasoning”

English translation is being prepared.

English translation is being prepared.

English translation is being prepared.

English translation is being prepared.

English translation is being prepared.

English translation is being prepared.

English translation is being prepared.

English translation is being prepared.

English translation is being prepared.

ARC Prize announces evaluation results of DeepSeek V4.1 Flash 1017 on ARC-AGI

ARC Prize announced DeepSeek V4.1 Flash 1017's results on the ARC-AGI series of benchmarks: 5.81% on ARC-AGI-3, 78.1% on ARC-AGI-2, and 94.5% on ARC-AGI-1, leading among open-weight models. Compared with the original DeepSeek V4.1 Flash released on September 10, 2026, 1017's ARC-AGI-3 semi-private set score rose from 2.68% to 5.81%, while its public set score fell from 11.05% to 8.97%. At maximum reasoning effort, the reasoning tokens used by 1017 for its first action decreased by an average of 36% across 25 public games and by 48.6% across 55 semi-private games.

ARC Prize announces evaluation results of DeepSeek V4.1 Flash 1017 on ARC-AGI

ARC Prize announces the verified results of DeepSeek V4.1 Flash 1017 on ARC-AGI: ARC-AGI-2 at 78.1% with $0.16 per task, ARC-AGI-1 at 94.5% with $0.06 per task, and ARC-AGI-3 at 5.8% with $1.2K under the provider adapter harness and 0.2% with $1.0K under the standard harness. The model leads open-weight models on ARC-AGI-3 and ARC-AGI-2, and matches the best score of open-weight models on ARC-AGI-1. Compared with the original V4.1 Flash from September 10, 2026, 1017's ARC-AGI-3 semi-private set score rose from 2.68% to 5.81%, while its public set score fell from 11.05% to 8.97%; at maximum reasoning effort, the reasoning tokens consumed by its first action decreased by an average of 36% across 25 public games and by 48.6% across 55 semi-private games.

ARC Prize announces evaluation results of DeepSeek V4.1 Flash 1017 on ARC-AGI

ARC Prize announces the evaluation results of DeepSeek V4.1 Flash 1017 on ARC-AGI: it scores 5.81% on the ARC-AGI-3 semi-private set, 78.1% on ARC-AGI-2, and 94.5% on ARC-AGI-1, leading among open-weight models. Compared with the original V4.1 Flash released on September 10, 2026, its ARC-AGI-3 semi-private set score rose from 2.68% to 5.81%, while its public set score fell from 11.05% to 8.97%. At maximum reasoning effort, the model's reasoning tokens consumed for the first action decreased by an average of 36% across 25 public games and by 48.6% across 55 semi-private games.

Building Efficient Agents with Knowledge Distillation on Amazon SageMaker AI

An AWS official blog post explains how to fine-tune a shopping agent on Amazon SageMaker AI using sequence-level knowledge distillation (SeqKD): using GLM-5.3 on Amazon Bedrock as the teacher model to generate trajectories, using the ShoppingBench scorer to retain only trajectories that solve the task, and then performing serverless SFT on the Qwen3.5-4B student model. On 324 held-out test questions, the distilled 4B student's average absolute success rate increased from 0.297 to 0.541, essentially on par with the teacher model and mid-sized frontier models; when the teacher's reasoning process was removed and only tool calls were learned, the average success rate dropped to 0.415.

ARC Prize publishes Meta Muse Spark Open's evaluation results on ARC-AGI

ARC Prize published the validation results of Meta's Muse Spark Open on ARC-AGI: ARC-AGI-1 at 89.0% with $0.29 per task, ARC-AGI-2 at 57.5% with $0.73 per task, and ARC-AGI-3 at 0.1% under the standard harness with $3.9K. ARC Prize states that its v1/2 scores are close to those of Haiku 5.5 and GPT-6 Luna, but the cost is about 9 times that of the latter; the model is priced on OpenRouter at $1.25 per million input tokens and $4.25 per million output tokens. As the reasoning tier increases, ARC-AGI-2 rises from 29.4% at medium (averaging 55k tokens/task) to 57.2% at xhigh (180k tokens/task).

ARC Prize tests Meta Spark Open: ARC-AGI-2 score 57.5%, cost about 9 times that of models with the same score

ARC Prize tested Meta Spark Open on ARC-AGI: it scored 89.0% on ARC-AGI-1, 57.5% on ARC-AGI-2, and only 0.4% on ARC-AGI-3. Its v1/v2 scores are close to those of Haiku 5.5 and GPT-6 Luna, but its cost is about 9 times theirs. Across providers on OpenRouter, the model costs $1.25/M for input and $4.25/M for output, while Haiku 5.5 and GPT-6 Luna both cost $0.10/M for input and $0.50/M for output. As the reasoning tier increases, ARC-AGI-2 rises from 29.4% at medium (averaging 55k tokens/task) to 57.2% at xhigh (180k tokens/task).

ARC Prize announces Meta Muse Spark Open's evaluation results on various versions of ARC-AGI

ARC Prize announced the validation results of Meta Muse Spark Open on ARC-AGI: ARC-AGI-1 is 89.0% at $0.29 per task, ARC-AGI-2 is 57.5% at $0.73 per task, and ARC-AGI-3 is 0.4% at $3.0K under the provider adapter harness and 0.1% at $3.9K under the standard harness. ARC Prize states that its scores on v1/v2 are similar to those of Haiku 5.5 and GPT-6 Luna, but the cost is about 9 times that of the latter; the model is priced on OpenRouter at $1.25 per million input tokens and $4.25 per million output tokens, while Haiku 5.5 and GPT-6 Luna are both $0.10 per million input tokens and $0.50 per million output tokens. On public tasks, as the reasoning scale increases, the ARC-AGI-2 score rises from an average of 55k tokens/task corresponding to 29.4% at the medium tier, to 150k corresponding to 51.0% at the high tier, and then to 180k corresponding to 57.2% at the xhigh tier; ARC-AGI-1 has already reached 92.8% at the high tier and rises to 94.0% at the xhigh tier.

Elorian Releases DecoyCount Benchmark: Frontier Models All Score Below 50% on Visual Counting Accuracy

Elorian and Mercor jointly released the DecoyCount benchmark, which tests whether models truly count objects or rely on visual shortcuts. Frontier models all scored below 50% accuracy, with the highest being GPT-6 Astra at 43.17%. Elorian says its visual reasoning model Glimpse scored 50.48% in a single run on 1,660 questions, making it the only model to exceed 50%, but this result comes from internal testing and did not use an agent or tool framework, whereas the six models in the paper each took the average of three runs. Elorian CEO Andrew Dai said the benchmark examines both vision and reasoning, and can distinguish whether a model failed to see the objects or miscounted them.

Amazon SageMaker AI Multi-Turn Reinforcement Learning Practical Guide: Training gpt-oss-20b with a Wordle Example

AWS has published a practical guide for multi-turn reinforcement learning (MTRL) on Amazon SageMaker AI. Using Wordle as an example, it deploys the agent and reward function to Amazon Bedrock AgentCore, and the training job updates LoRA adapters via GRPO. In the example, gpt-oss-20b at default reasoning effort improved its solve rate on held-out words from 61% to 84%, at a training token cost of about $339 and an AgentCore runtime cost of about $79. The article also details failures such as two inverted reward design errors, token invalidation under concurrency, and tool calls degrading into plain text, and recommends evaluating under the decoding settings used in final serving.

DeepSeek Search ranks 17th on the Artificial Analysis Search Index

DeepSeek Search scores 69 on the Artificial Analysis Search Index, ranking 17th among 27 search products, and is the lowest-priced paid search API on the list. It charges $0.75 per 1k searches, with a per-task cost of about $0.032, ranking 5th cheapest among 27 products. On AA-Omniscience its accuracy is 70%, ranking 10th, but on BrowseComp, which requires multi-hop reasoning, it is only 63%, ranking 25th; each search returns about 7k tokens, raising model cost to the 2nd highest on the list.

English translation is being prepared.