Search “Hugging Face”

How VocabX cuts real-world inference costs by 77% with specialized SLM vocabularies

VocabX selects tokens by cost rather than frequency and simultaneously rebuilds subword segmentation and multi-word merging, compressing the vocabulary of domain-specific small models to a smaller scale and reducing inference costs by 77% in real-world applications. Taking a scoring model as an example, generating one token costs 7.4 times as much as reading one token, and the output-layer vocabulary shrinks from about 152,000 rows to about 65,000 rows. This method targets the memory bandwidth bottleneck of small models such as Qwen3-4B at medium batch sizes, whereas methods such as FVT, MWT, and in-place BPE select tokens only by frequency and address only a single level.

Profiling TorchNeuron (Part 1): System-Level Performance Profiling on AWS Trainium

AWS has released Part 1 of the TorchNeuron profiling guide, which explains how to use NeuronConfig and NeuronProfiler for system-level profiling on AWS Trainium, and how to view system traces, device timelines, and per-engine breakdowns through Neuron Explorer. Taking RMSNorm as an example, a naive PyTorch implementation has a total device time of 2.355 ms, while torch.rms_norm(), fused into a single Neuron kernel, reduces this to 403.3 μs.

Deploying the RFD4-Proteina model on Databricks

NVIDIA and Databricks plan to integrate NVIDIA BioNeMo's open-source protein design model RFD4-Proteina into Databricks' Genesis Workbench, running on the now generally available AI Runtime serverless NVIDIA GPU compute. The model can simultaneously generate protein sequences and all-atom structures, and supports optional reward-guided search to steer candidate designs toward target properties such as thermostability and binding affinity. Genesis Workbench includes two built-in RFD4-Proteina workflows, Antibody Design and Vaccine Design, and can be extended through Genie Code and NVIDIA BioNeMo Agent Toolkit skills.

English translation is being prepared.

English translation is being prepared.

Nvidia adds Agent Knowledge Foundation to Agent Toolkit

At GTC Berlin, Nvidia added Agent Knowledge Foundation to Agent Toolkit, packaging the Nemotron Retriever models, the new Auto-Ontology library, the Kumo family of models for structured data, and CUDA-X data processing libraries; data remains in place and is managed under enterprises' existing permissions. Nvidia says Nemotron Retriever improved search relevance by 65% in tests in the open-source OpenSearch environment, SAP's fine-tuned reranker runs nearly three times faster, and ServiceNow's reranking delivers better quality with latency reduced by nearly 25%. Auto-Ontology provides a REST API and an MCP server for agents to query, and Nvidia says it does not compete with data platforms such as Snowflake but rather partners with them.

NVIDIA GTC Berlin 2026 announces 12 releases, covering AI factories, telecommunications, and enterprise agents

At GTC 2026 in Berlin, NVIDIA announced 12 releases, covering AI factory deployment, semiconductor manufacturing expansion, telecom AI-RAN, enterprise agents, robotics and autonomous driving, as well as new open models for enterprise data, protein design, and quantum computing. On the infrastructure side, the DSX POD modular AI factory reference design is said to shorten deployment time from six months to as little as 14 days, and DSX MaxLPS is said to deliver equivalent AI compute using up to 30% less power, with partners planning to deliver in early 2027. On the telecom side, T-Mobile completed what it calls the world's first AI-RAN call on a commercial network in Bellevue, Washington, with NVIDIA and Nokia, and Elisa completed Europe's first AI-RAN field trial; on the enterprise side, it expanded the open-source Agent Toolkit and introduced the OpenShell reference architecture for monitoring and restricting agent behavior.

English translation is being prepared.

DeepSeek open-sources the DeepInfer inference framework, processing 155 trillion tokens per day in a single cluster

DeepSeek has open-sourced DeepInfer, its self-developed inference framework that supports DeepSeek-V4.1-Flash, and published production data: a single cluster processes 155 trillion tokens per day, 96% input cache hit rate, 400ms first-token latency, and 230 tokens/s generation speed. The framework is customized based on vLLM and supports distributed inference, speculative decoding, SWA Bounded Replay, and structured output, with target platforms NVIDIA SM100 and Huawei Ascend 950. DeepSeek also states that components such as KV cache management and load balancing are not yet fully open-sourced, while the vLLM team says that under a matching GB200 configuration, decoding throughput is within 6% of the results published by DeepInfer.

DeepSeek open-sources the DeepInfer inference framework, processing 155 trillion tokens per day on a single cluster

DeepSeek has open-sourced DeepInfer, the production inference framework supporting V4.1 Flash, and published production data including 155 trillion tokens processed per day on a single cluster, a 96% input cache hit rate, an average 400ms time to first token, and 230 tokens/s generation. The framework is customized based on vLLM and uses the Apache 2.0 license, supports distributed inference, speculative decoding, SWA Bounded Replay and structured output, and targets NVIDIA SM100 and Huawei Ascend 950. DeepSeek says components such as KV cache management and load balancing are not fully open-sourced because they are coupled with internal infrastructure, while the vLLM team reports that under matched GB200 configurations, decoding throughput differs from DeepInfer's published results by less than 6%.

English translation is being prepared.

English translation is being prepared.

English translation is being prepared.

English translation is being prepared.

Zyphra Inference Wave 2: Accelerating MoE and Long-Context Inference Serving on MI355X

Zyphra has released ZOE, its second-generation inference optimization, which on AMD Instinct MI355X reduces GLM-5.3's time to first token (TTFT) at 512K context from 178.8 seconds to 64.6 seconds (2.8x), and from 42.3 seconds to 11.8 seconds (3.6x) at 90% prefix cache hit rate. Its CSA top-K selection HIP kernel raises DeepSeek-V4-Pro's single-user decode throughput at 1M context from 43.5 to 61.1 tokens/s (1.40x). AMD MXFP4 quantization of GLM-5.3 shrinks the model from 756 GB to 410 GB, with benchmark scores such as AIME25 within 3 points of FP8.

English translation is being prepared.

English translation is being prepared.

METR reviews the misalignment risk chapter of Anthropic's August 2026 risk report

METR conducted an external review of the "Misalignment in high-risk scenarios" chapter in the August 2026 edition of Anthropic's risk report. It agreed that the catastrophic misalignment risk of the covered models (including Mythos 5 and the internal model Model 2) is low, but held that this is mainly due to limited model capabilities rather than safeguards and alignment training. METR pointed out that the covert capability benchmark in the report cannot represent a model's ability to evade Anthropic's internal monitoring systems, and stated that recent events weaken the report's argument that unknown severe situational misalignment is "unlikely." METR also recommended disclosing four redacted pieces of information and conducting end-to-end red-team testing of the monitoring system.

Google integrates TorchTPU into TorchTitan to optimize model training for TPUs

Google integrates TorchTPU into Meta's TorchTitan; Cloud TPU can now train PyTorch models directly through the native backend torch.device("tpu"), without rewriting them into JAX. The TorchTitan TPU branch provides pretraining recipes for Llama 3 70B/405B, Qwen 3 8B/32B/235B MoE, and FLUX.1 12B, and replaces standard implementations with Pallas kernels such as Splash Attention. On FLUX, SimpleFSDP brings TorchTitan to 99% performance parity with JAX MaxDiffusion.

English translation is being prepared.