35B Multimodal on 3 GB of RAM: MiniCPM-V 4.7 and the Trick That Made It Work

Verdict: OpenBMB's MiniCPM-V 4.7 is the first open-source MoE multimodal model that genuinely runs on a phone — 13+ tokens/s on an iPhone 17 Pro Max with about 3 GB of memory, Apache-2.0, with day-0 support in llama.cpp, Ollama, vLLM and SGLang. The benchmark table is not the story. The story is Lookahead Expert Scheduling (LES), a 5× speedup on real devices — and it's the first time I've seen someone attack the actual bottleneck of on-device MoE, which is memory bandwidth, not compute. If you build privacy-sensitive mobile features that read documents, receipts or screenshots, this is the first on-device option where you don't have to choose between "small enough to ship" and "smart enough to matter."

Why running a MoE on a phone was broken

A MoE model has most of its brains offline. The 35B-A3B naming means 35B total parameters with only ~3B active per token — great for compute, terrible for memory, because you can't predict which of the dozens of experts a token will route to until the routing layer fires. The two existing workarounds each give something up: keep every expert in RAM (needs a device most people don't own) or preload a fixed subset of experts (runs fast, but the model quietly gets dumber). Neither path ever felt shippable.

LES takes the third road. Its cross-layer routing prediction watches what the current layer is doing and preloads the experts the next layer will probably need — from flash, before they're asked for. Per OpenBMB's README, this delivers a 5× speedup on real devices without sacrificing model capability. That last clause is the whole point: capability stays intact because the right experts are simply already there when routing fires.

The rest is sensible engineering stacked on top: per-layer mixed-precision quantization and a contiguous storage layout for expert weights, plus DSpark speculative decoding with dedicated draft heads for both the 1B and the 35B-A3B variant.

The numbers, checked

From OpenBMB's official README: 13+ tokens/s decoding with ~3 GB of memory on iPhone 17 Pro Max (12 GB RAM device); 5× LES speedup on real devices; Apache-2.0 license; iOS, Android and HarmonyOS deployment with day-0 adapter support for Rockchip, Intel, AMD and Arm platforms. Chinese coverage by 量子位 adds context numbers I couldn't independently verify against the official table: CMMLU 90.21 and τ2-Bench 84.65 for the 35B-A3B, both claimed as best-in-table, and 50M+ cumulative downloads for the MiniCPM series as of September 2026.

One thing the headline hides: 3B active parameters is a 3B-class brain. The speed comes from activating little; the quality comes from routing well. For OCR-style extraction, chart reading, receipt parsing and grounded Q&A over images, that's plenty — the in-context demos are exactly this class of task. For long-horizon reasoning or writing that needs a large working memory, you'll still want a cloud model. I haven't run the model myself; these are official figures and third-party reported scores, and real-device results will vary by quantization choice.

Who should care

Ship it if you build mobile features where the data can't leave the device: contract and report reading, expense receipt processing, field notes, screenshots. The memory math finally works — a 12 GB flagship can host a 35B model in 3 GB and keep 9 GB for everything else, which matters more than usual this year since RAM prices keep climbing and 12 GB is now the entry bar for "AI phones."

Wait if your use case needs top-tier reasoning, or you expect consistent performance across low-end Android devices — 3 GB of model weight still assumes a 12 GB-class phone. And if you were considering the desktop self-hosting route instead, our DeepSeek 4.1 Flash deployment guide covers the server-side answer, which remains a different (and easier) problem.

Where this fits in the on-device race

This is the second big "absurd model on consumer hardware" result this month, and the two point in opposite directions. Qwen's MoE line keeps getting bigger — we covered running a 80B model on 4 GB of RAM via aggressive offloading last week. MiniCPM-V 4.7's approach is smarter: instead of pretending memory doesn't exist, it makes flash-resident experts fast enough that the RAM question mostly goes away. My bet: the offloading tricks age out with every hardware cycle, but routing-prediction prefetch composes with any future device — which is why this release matters more than its benchmark table.

Caveat (Oct 17, 2026): figures are as published by OpenBMB on October 13–15, 2026; on-device speeds depend on device, quantization and framework. Third-party context numbers are credited to 量子位's coverage and not independently verified.