Self-hosting

Written 13 Sep 2026 · Prices verified 30 Oct 2026

Running models on your own hardware: VRAM math, quantization trade-offs, and when local actually beats the API.

• OXR-1.0: The 1.3B Document Parser Topping Three Benchmarks — Deployment Facts and License Strings Before You Ship ItOXR-1.0 ranks #1 on olmOCR-Bench, OmniDocBench and DocParsingBench with an RSI post-training loop — but the license isn't plain Apache-2.0 ($2M threshold, attribution duty, no competing OCR API) and the official VRAM floor is 24GB, not the RTX 3060 you may have read. Deployment facts, license strings, and honest limits.
• MOSS-Image Is the Open-Weight Image Model You Can Actually Fine-Tune at Home — but Budget for the 9B Text Encoder, Not the 5B DiTMOSS-Image (OpenMOSS, Apache-2.0) topped Hugging Face trending in four days. The local deployment math: 5B DiT plus a 9B Qwen3.5 text encoder, LoRA fine-tuning from 6GB VRAM, day-0 ComfyUI/SGLang/vLLM support, and the community quantization ecosystem. What GPU you actually need.
• Xing4.0-29B-Intent Locally: 62 GB of Weights, No GGUF, and One Honest AlternativeChina Telecom's Xing4.0-29B-Intent is Apache-2.0 and on Hugging Face — but it's 62.43 GB of BF16 with no official GGUF. Hardware math, deployment paths, and why fine-tuning on its 2.75M corpus beats running it.
• 35B Multimodal on 3 GB of RAM: MiniCPM-V 4.7 and the Trick That Made It WorkOpenBMB's MiniCPM-V 4.7 35B-A3B hits 13+ tokens/s on an iPhone 17 Pro Max using just 3 GB of memory. The headline number is less interesting than Lookahead Expert Scheduling — the first engineering answer to running MoE experts off flash without crippling the model. Here's how it works and who should actually ship with it.
• 80B Models on 4GB RAM: The Real NumbersSwiftlet runs Qwen3-Next-80B in 4.3GB of RAM at 4.5-5 tok/s on an M5 Mac. An iPhone 17 manages 1 tok/s. AirLLM fits 70B in 4GB VRAM with no quantization and publishes no speed at all. The numbers that decide whether any of this is usable.
• DGX Station for Windows: Trillion Parameters on a Desk, Price TBDNVIDIA's DGX Station for Windows ships Q4 2026 with 748GB unified memory and 20 PFLOPS FP4 — enough for trillion-parameter MoE models locally, not a 1T dense model, and the math matters. Who the six-OEM desktop supercomputer is actually for, what OpenShell changes for agent security, and why the unannounced price is the whole decision.
• Self-Hosting GLM-5.3-Flash: $6,000 of Hardware vs a $125/Month APIThe honest math on self-hosting GLM-5.3-Flash: dual DGX Spark at 4-bit (~164 GB, 29-70 tok/s) costs about $6,000 up front - roughly four years of heavy API use at $0.15/$0.50 per M. Hardware paths, quants, and where the API wins.
• Ollama v0.40.0-rc0: Local Models Now Run on MLX by Default on Apple SiliconOllama v0.40.0-rc0 (25 Sep 2026) makes supported model architectures run on MLX by default on Apple Silicon — no flags needed. What changes, real MLX vs llama.cpp numbers from community tests, and why stable-channel users may want to wait. RC caveats: docs lag, partial model coverage, no fallback switch.
• How to Deploy DeepSeek 4.1 Flash — Including the Part That Deployed ItselfOne model string, vision in the main model, 552B open weights, retired names already routing to it — and the Sept 14 Pro question. Checked 12 Sep 2026.