Researchers Use GPT-6 Astra to Directly Rewrite Qwen3.5-0.8B Weights to Improve Math Ability
Yuchen Jin paraphrases Rikiya Takehi's experiment: using GPT-6 Astra to directly edit the weights of Qwen3.5-0.8B to improve math ability, changing only 0.016% of the weights fixed the model's reasoning loops, improving math ability by up to 6x and instruction following by 3x, all without using gradients or doing any training. The author calls this method Weight-Space Programming, believes that frontier LLMs can now natively read and write neural network weights, and says it goes a step further than knowledge distillation.
Why read this
SelectedAI score 68
This paraphrase demonstrates a new approach in which a large model directly rewrites the weights of a small model, allowing readers to understand a parameter-editing idea that does not rely on gradient training.
Topics
View original postWe asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math. It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training. We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵 Blog: https://t.co/bOWCcFhCVq
View original
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math. It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training. We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵 Blog: https://t.co/bOWCcFhCVq
Body
This is cool as heck!
Imagine using a large LLM to directly edit a smaller model's weights and dramatically improve its capabilities.
It's like gene editing for neural networks. Even cooler than distillation!