Fireworks publishes GLM 5.2 numerical alignment experiment
On September 30, 2026, Fireworks published a GLM 5.2 numerical alignment experiment. The experiment shows that when running the same reinforcement learning task on GLM 5.2, an inconsistency between the trainer and the rollout engine in how token probabilities are computed leads to a KL of about 0.013, roughly 45% of samples discarded per batch, and the reward dropping from about 0.9 to below 0.2 around step 20; after numerical alignment, the KL is 0 and the reward remains stable throughout. The same material also documents a Qwen3.5-MoE experiment: precision differences in the weighted sum of expert outputs brought the k3 of image tokens to 0.296, about 300 times the equivalence threshold, and k3 dropped to zero after merely replacing the MoE module with the Hugging Face reference implementation. Currently, this experiment is published by Fireworks as a first-party source, and no subsequent independent progress has emerged.
ent, the KL is 0 and the reward remains stable throughout. The same material also documents a Qwen3.5-MoE experiment: precision differences in the weighted sum of expert outputs brought the k3 of image tokens to 0.296, about 300 times the equivalence threshold, and k3 dropped to zero after merely replacing the MoE module with the Hugging Face reference implementation. Currently, this experiment is published by Fireworks as a first-party source, and no subsequent independent progress has emerged.
Matched fieldsSummary