SenseTime Open-Sources SenseNova Loop-9B-Preview Looped Computation Model
SenseTime has open-sourced SenseNova Loop-9B-Preview, which adopts the Looped Transformer architecture, retains the original decoder modules of the first and last 4 layers each, and replaces the middle layers with parameter-sharing looped computation modules, expanding the effective computational depth through multiple rounds of iteration and benchmarking against 27B models at the 9B parameter scale. The model introduces mHC (Manifold-Constrained Hyper-Connections) multi-path residual connections to enhance feature interaction; in ablation experiments, GPQA Diamond and TeamBench improved by 4.42 points and 13.34 points, respectively. SenseTime claims that the model outperforms other models in code generation, office scenarios, and agent tasks, and surpasses Qwen3.5-27B on WorkBuddy Bench, RaccoonPPT Bench, and AutomationBench. The model weights have been open-sourced.
Why read this
AI score 60
SenseTime has turned its looped computation architecture into 9B open-source weights that can be downloaded directly, allowing readers to observe the path by which small-parameter models catch up to large models through computational depth.
Topics
Body · AI translation
SenseTime has open-sourced SenseNova Loop-9B-Preview. This is SenseTime's frontier exploration into the next-generation model architecture: it adopts an innovative Looped Transformer architecture, introduces recurrent computation to extend the model's effective computational depth, and explores a more thorough information interaction mechanism to enhance the model's ability to handle complex tasks, matching 27B-parameter models at a limited parameter scale of only 9B.
From parameter scale to computational depth
As large model capabilities continue to evolve, improving model capability is no longer limited to the single path of scaling up parameter counts. How to further unleash model potential by increasing effective computational depth while controlling parameter scale is becoming a direction worth exploring. Recently, recurrent computation has become a key direction in large model architecture research, and research communities at home and abroad have carried out exploratory work around Looped Transformer. However, current explorations mostly remain at the academic research level, and the community still lacks a Looped Transformer model that can be directly used in everyday settings.
The open-sourced SenseNova Loop-9B-Preview provides the community with an open-source practice of a 9B-level Looped Transformer, filling this gap and reflecting SenseTime's continuous exploration of and thinking about next-generation large model architectures.
However, how to truly translate multi-round computation into improvements in model capability, especially in aspects such as information transfer, feature interaction, and training stability, still leaves room for further exploration. Existing related work is still mainly research validation, and how to balance computational depth, model capability, and practical application needs at larger model scales also remains to be further advanced.
Recurrent structure and information interaction mechanism
SenseNova Loop-9B-Preview is based on a 9B-scale model, retaining the original decoder modules of 4 layers each at the beginning and end, and transforming the middle layers into parameter-sharing recurrent computation modules, extending effective computational depth through multiple rounds of iteration.
On this basis, the model further explores the information transfer mechanism in recurrent computation, attempting to introduce mHC (Manifold-Constrained Hyper-Connections), using multi-path residual connections and constrained information mixing to enhance feature interaction and maintain the stability of signal propagation. The overall design aims to explore a new path for improving model reasoning capability while controlling parameter scale.
Model performance: showing potential across multiple types of complex tasks
While maintaining a 9B-level parameter scale, SenseNova Loop-9B-Preview demonstrates strong cross-task performance. Under the current evaluation settings, the model outperforms other models in multiple directions including code generation, office scenarios, and agent tasks, and further surpasses Qwen3.5-27B in evaluations such as WorkBuddy Bench, RaccoonPPT Bench, and AutomationBench. These results preliminarily demonstrate the potential of improving small-parameter models' ability to handle complex tasks by extending effective computational depth.
SenseTime further evaluated the actual benefits of mHC through ablation experiments. After adopting this mechanism, the model achieved significant performance improvements, with GPQA Diamond and TeamBench increasing by 4.42 points and 13.34 points respectively, demonstrating application potential in complex reasoning tasks and multi-step Agent tasks. In the future, mechanisms such as DSA and adaptive iteration counts will continue to be explored to improve the model's reasoning efficiency and task performance.
Continuing to explore next-generation model architectures
SenseNova Loop-9B-Preview is only a starting point. Toward the next-generation SenseNova, SenseTime will further explore the scaling potential of Looped Transformer at larger model scales and the adaptive "fast and slow thinking" mechanism, deeply tap more potential of parameter sharing, and continue to break through the boundaries of model capability and reasoning efficiency with more flexible computation methods and higher parameter efficiency.
Model weights open-sourced: