Understand this story
On November 4, 2026, Together AI released Polymorphic Inference, whose first implementation is Together Compiler. According to the Together AI blog, the solution is described by the company as the world's first production-grade recursive self-improving (RSI) inference system, and has used AI agents to optimize 40+ models on its endpoints, replacing static tuning of inference engine configurations with Continuous Compilation, in order to cope with accelerating model releases, diversifying architectures, changing traffic patterns, and hardware iterations from B200 to GB300. On the same day, Unite.AI reported that the compiler takes traffic patterns, models, and hardware as inputs, has parallel agents generate and measure candidate optimizations, and uses hot reloading to avoid restarting the entire engine; in tests on Kimi K3, GLM 5.3, MiniMax M3.1-Flash, and DeepSeek v4.1-Flash, it reached 1.3x to 3x the tokens per second of a highly optimized open-source baseline within 24 hours. Customers can apply for early access through a limited beta.
AI-generated from public reports
Latest development · Report
Together AI Launches Polymorphic Inference, Continuously Compiling and Optimizing Production InferenceDevelopments
1 confirmed developments
- Source:Together AI Blog
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Nov 4
Unite.AI
Together AI Launches Polymorphic Inference, Continuously Compiling and Optimizing Production Inference
On November 4, Together AI released Polymorphic Inference, whose first implementation is Together Compiler, which uses agents to continuously re-tune inference for changing models, traffic, and hardware. The company says that in tests on Kimi K3, GLM 5.3, MiniMax M3.1-Flash, and DeepSeek v4.1-Flash, it achieved 1.3x to 3x the tokens per second of a highly optimized open-source baseline within 24 hours. The compiler takes traffic patterns, models, and hardware as inputs, has parallel agents generate and measure candidate optimizations, avoids restarting the entire engine through hot reloading, and customers can apply for early access through a limited beta.
Together AI BlogFirst-party
Polymorphic Inference: Together AI Uses AI Agents to Self-Optimize Open-Source Model Inference
Together AI has launched Polymorphic Inference, which it calls the world's first production-grade recursive self-improvement (RSI) inference system, having used AI agents to optimize 40+ models on its endpoints. The solution addresses accelerating model releases, architectural diversification, changing traffic patterns, and hardware iterations such as B200 to GB300, replacing statically tuned inference engine configurations with Continuous Compilation.
Heat trend for this story
- Current
- 7
- Period peak
- 16 Nov 4 23:00
- 24-hour change
- -50%
Move the pointer or select the chart to inspect hourly heat. Use the left and right arrow keys with a keyboard.