Rank #04 · Daily · AI
colibri: Pure-C Engine That Streams MoE Experts From Disk
"Tiny engine, immense model." It runs hundreds-of-billions-parameter MoE models on the computer you already own, no GPU required.
What it does
A MoE model activates only a small slice of its weights for each token. colibri takes advantage of that: it keeps the shared, always-used weights resident in RAM and streams the routed experts from disk when the router asks for them. The README's example is GLM-5.2, with roughly 744B parameters, about 40B used per token, and only about 11 GB of that changing between tokens. Its sample session shows GLM-5.2 int4 streaming on CPU with about 9.9 GB resident.
- Pure C, one file per model family, zero dependencies.
- Thirteen engines listed today, mostly language models (GLM, Kimi, DeepSeek, Qwen, MiMo, OLMoE families and others), plus one image model and decision models.
- Reads existing GGUF files as-is. Runs on CPU, on any GPU via Vulkan, or on NVIDIA via CUDA.
- One-step setup (
START-HERE.bat/./start-here.sh) checks RAM, disk, CPU, and GPU, recommends a model that fits, builds or downloads the engine, downloads the model with resume support, and opens a dashboard. - Exposes OpenAI- and Anthropic-compatible local endpoints, so agents and apps can connect to it.
Minimum per the README: 8 GB RAM (16 GB+ recommended) and about 22 GB of free disk for the smallest model. For the large streamed models, disk speed sets the token rate, so a fast NVMe drive matters most.
Who it's for
Local-LLM users who want to try frontier-scale open MoE models without a multi-GPU rig, along with tinkerers who appreciate a small, readable C codebase. If you need high throughput, this isn't the tool. Streaming from disk trades speed for reach.
Stars / language / license
46,552 stars · C · Apache-2.0
Data verified: 2026-11-03 09:04 Asia/Shanghai (GitHub API get_repository + README get_file_contents)
Trending note: +675 stars today (Trending selection captured 09:02 Asia/Shanghai)
Category tags
AI · local-LLM · MoE · inference · C · GGUF · edge
How it differs from similar projects
llama.cpp has broad model coverage and can mmap weights, but it's optimized around fitting the model in RAM or VRAM. colibri is built specifically around expert streaming for huge MoE models, with per-family engines. FreeToken (#05 today) also targets big MoE on desktops, but it's a Python serving stack focused on NVIDIA GPU/CPU co-execution. colibri is dependency-free C that runs CPU-first. The README publishes per-machine tok/s tables. Those are the project's own figures and TidyHow hasn't reproduced them. Model weights keep their upstream licenses.