Understand this story
Baseten's engineering blog states that it rebuilt the incremental weight synchronization path for the approximately 770GB checkpoint of GLM-5.3, reducing average sync time from about 36 seconds to 15 seconds and request pause time from 6 seconds to 0.8 seconds. The new design uses the weights on the GPU as the parent copy, no longer retaining a full copy in CPU memory, and uses Adler-32 checksums to perform constant-time updates on changed bytes. The freed host memory is used for KV cache offloading; at a concurrency of 48, the cache hit rate rose from 2% to 92%, and single-GPU output throughput increased 3.2 times. The same blog also mentions that Kimi K3's 8.4GB incremental sync takes 75 seconds, with a 4.1-second pause.
AI-generated from public reports
Latest development · Report
Baseten cuts 770GB incremental weight sync from 36 seconds to 15 secondsDevelopments
1 confirmed developments
- Source:English text is being prepared.
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Oct 31
Baseten Engineering BlogFirst-party
Baseten cuts 770GB incremental weight sync from 36 seconds to 15 seconds
Baseten's engineering blog states that incremental weight synchronization for the approximately 770GB GLM-5.3 checkpoint dropped from an average of about 36 seconds to 15 seconds, and request pauses fell from 6 seconds to 0.8 seconds. The new design uses the weights on the GPU as the parent copy, no longer keeps a full copy in CPU memory, and uses Adler-32 checksums to perform constant-time updates on changed bytes. The freed host memory is used for KV cache offloading; at a concurrency of 48, the cache hit rate rose from 2% to 92%, and per-GPU output throughput increased 3.2x. For Kimi K3, the 8.4GB incremental sync takes 75 seconds with a 4.1-second pause.