Understand this story
On November 4, 2026, Together AI released Together Compiler (TKCC), a compilation system that uses high-throughput agents to continuously optimize inference workloads. The company claims that within less than 24 hours it achieved TPS improvements on frontier open-source models such as Kimi K3, GLM 5.3, MiniMax M3.1-Flash, and DeepSeek v4.1-Flash; two official blog posts respectively state improvements of up to 2.6x and 1.3-3x. On SemiAnalysis's InferenceX low-latency benchmark, the system achieved current best results of 1111 TPS for MiniMax M3.1-Flash and 1205 TPS for DeepSeek v4.1-Flash, and in the Llama 1B batch size 1 scenario it is 1.7x faster than the open-source megakernel. TKCC relies on a persistent engine to hot-load candidate forward computations and on a two-tier fast/slow engine to balance throughput and fidelity; its experimental throughput is 252x higher than that of baseline agent-based optimization systems, and it is currently used to serve trillions of tokens per day.
AI-generated from public reports
Latest update · Discussion
MarkTechPost joined this discussionDevelopments
1 confirmed developments
- Source:Together AI Blog
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Nov 5
MarkTechPostRelated discussion
MarkTechPost joined this discussion
This post contributes to heat; it is not a formal report or confirmed fact.
Nov 4
Together AI BlogFirst-party
Together AI Releases Together Compiler, Using Agents for Continuous Compilation and Inference Optimization
Together AI has released Together Compiler (TKCC), a compilation system that uses high-throughput agents to continuously optimize inference workloads, achieving up to 2.6x TPS improvement within less than 24 hours on frontier open-source models such as Kimi K3, GLM 5.3, MiniMax M3.1-Flash, and DeepSeek v4.1-Flash. On SemiAnalysis's InferenceX low-latency benchmark, it achieved current best results of 1111 TPS for MiniMax M3.1-Flash and 1205 TPS for DeepSeek v4.1-Flash, and in the Llama 1B batch size 1 scenario it is 1.7x higher than the open-source megakernel. TKCC relies on a persistent engine to hot-load candidate forward computations and a fast/slow two-tier engine to balance throughput and fidelity; its experimental throughput is 252x higher than baseline agent optimization systems, and it is currently used to serve trillions of tokens per day.
Together AI BlogFirst-partySelected
Together AI Releases Together Compiler, Boosting Inference Throughput by 1.3-3x
Together AI has released Together Compiler (TKCC), which uses high-throughput agents to continuously generate inference optimizations for different traffic patterns, models, and hardware. The company claims that in under 24 hours it achieved 1.3-3x TPS improvements on Kimi K3, GLM 5.3, MiniMax M3.1-Flash, and DeepSeek v4.1-Flash, and achieved SOTA on SemiAnalysis's InferenceX low-latency benchmark with MiniMax M3.1-Flash at 1111 TPS and DeepSeek v4.1-Flash at 1205 TPS. The system employs a resident engine and a fast/slow two-tier engine to balance experimental throughput and measurement fidelity; its experimental throughput is 252x higher than that of baseline agent systems, and it is already in use for online serving.
Heat trend for this story
- Current
- 12
- Period peak
- 14 Nov 5 17:00
- 24-hour change
- —
Move the pointer or select the chart to inspect hourly heat. Use the left and right arrow keys with a keyboard.