Claude Code integrates message threads, reducing first-token latency for long requests by about 35%
After Claude Code integrated message threads, on requests of 500K to 1M tokens, the median first-token latency dropped by about 35%; in long conversations, the median size of a single request fell from about 800 KB to 1.8 KB. Message threads have entered public beta in the Messages API. The server caches the conversation between turns, so callers only need to send the new message and the ID of the previous message, and must add the thread parameter and beta header. It is available on Claude Platform, Claude Platform on AWS, Claude in Microsoft Foundry, and Claude on Google Cloud, and is billed at standard token prices.
Why read this
SelectedAI score 60
The official figures show the changes in first-token latency and request size for message threads on long-context requests, which can be used to assess the engineering benefits in long-conversation scenarios.
Topics
Body
We recently integrated message threads in Claude Code. On requests with 500K to 1M tokens, median time to first token is about 35% lower.
In long conversations, the median request went from about 800 KB to 1.8 KB.