MiniCPM-V 4.7 achieves 20.51 tokens/s decoding on iPhone 18 Pro Max

ModelBest's MiniCPM-V 4.7 achieves an average decoding speed of 20.51 tokens/sec on iPhone 18 Pro Max (12GB), a 6.07x speedup over the unoptimized SSD expert-loading baseline on the same device. This speed comes from Lookahead Expert Scheduling (LES), optimized expert loading, fine-grained quantization, and DSpark speculative decoding. Follow-up reports further explain that the approach prefetches experts likely needed by the next layer via cross-layer routing prediction, loads remaining experts in parallel, and performs contiguous storage reordering and layer-wise mixed-precision quantization of expert weights, together with DSpark speculative sampling; the model is a MoE architecture with 35B total parameters and 3B activated per token, with the original model architecture and expert routing unchanged and no additional training required. The model is open-sourced on GitHub and Hugging Face.

AI-generated from public reports

1. Oct 20· 2 reports

   English translation is being prepared.

   Source:English text is being prepared.