The 744B Model That Lives in 9.9 GB and Reads Itself Off Your Disk

There is a line in the colibri README that reads like a typo. A 744-billion-parameter model, int4, streaming from CPU, ready in 32 seconds, 9.9 GB resident. On a laptop. With no graphics card.
It is not a typo, but it is also not the whole sentence. The 9.9 GB is what stays in memory. The 429 GB of experts that the model actually consults live on your disk, and get read in as the model asks for them. That distinction is the entire project.
colibri is written in C, one file per model family, by a developer who goes by JustVugg. It hit GitHub trending this week at roughly 803 stars a day. Thirteen engines ship today: ten for language models, one that draws pictures, and two that make classification decisions. The pitch is unusually blunt about the tradeoff, which is why it is worth taking seriously.
What streaming experts actually buys you
Mixture-of-experts models are already doing this trick internally. A 744B model does not use 744B parameters for a token. It routes that token to a handful of experts, maybe 8 out of 512, plus a shared one. The rest sit idle.
Data-center inference keeps all of them in VRAM because VRAM is faster than any alternative. colibri inverts the assumption: if 95% of the weights are idle on any given token, why keep them in the most expensive memory you own? Put the always-used part in RAM, put the experts on disk, and read only the ones the router picks.
The catch is latency, and colibri does not hide it. In the README's own table, GLM-5.2 runs at 0.05 to 0.1 tokens per second cold on a 25 GB laptop. That is not a chat experience. It is a sentence every twenty seconds. On a 128 GB Ryzen AI Max+ 395 it reaches 1.83 tok/s. On six RTX 5090s, 9.0 to 9.2. DeepSeek V4 Flash, the 284B one, gets 0.93 tok/s on a Ryzen 7 5800X with 32 GB, and 1.5 to 1.6 with an RTX 5080 helping.
Those numbers are the honest part of the project. A lot of "runs on your laptop" claims quietly mean "runs on your laptop if your laptop is a workstation." This one publishes the slow case next to the fast one, and the slow case is genuinely slow.

The setup is a batch file
What makes colibri more than a curiosity is that someone did the unglamorous work of packaging it for people who do not compile things.
On Windows you download the ZIP, double-click START-HERE.bat, and it offers to install Python if you do not have it. On Linux and macOS it is a git clone and ./start-here.sh. The script then does five things in order: reads your RAM, disk, CPU and GPUs; recommends a model whose resident portion plus a minimum expert cache fits in your RAM; builds the engine for your machine if a compiler exists, otherwise downloads a prebuilt binary; downloads the model with resume; and starts a local server with a dashboard.
Minimum spec is 8 GB of RAM and 22 GB of free disk for the smallest model. A GPU is optional, and the README is specific about when it helps: on a discrete card always, on an integrated GPU only for the three models they measured as faster there.
The server speaks two dialects that matter. http://127.0.0.1:8000/v1 is an OpenAI-compatible base URL, and the bare port is Anthropic-compatible. So the practical use is not typing into a terminal. It is pointing an existing tool at localhost and letting it think it is talking to a cloud provider.
It also draws pictures
The thirteenth engine is the one that caught my attention, because it turns a text-inference project into something relevant to image work. colibri ships a Qwen-Image-2.1 engine. Same 7B model Alibaba released in September, the one with native 2048×2048 output and a real alpha channel.
The measured number in the README: 33 GB download, 12 to 18 GB of RAM, Vulkan for the GPU path, and one 768×512 picture in two minutes forty seconds on eight Zen 4 cores, CPU only. You can also hit it over HTTP at POST /v1/images/generations, and coli chat will draw the result inside the terminal using block characters.
Two minutes forty for a 768×512 image is not going to replace anybody's API bill. But it reframes what the model is. Qwen-Image-2.1 shipped with a research-only license, which means the moment you put it behind a paid product you have a licensing conversation to have. A local engine that runs it on the CPU is exactly the shape of a tool for the phase before that conversation: you are evaluating, not selling.
Where the disk becomes the bottleneck
The README says it plainly: for the large models the disk sets the speed, and a fast NVMe helps most. That is a real architectural constraint, not a footnote. Sequential read throughput on a SATA SSD is roughly a tenth of a decent NVMe drive, and the access pattern here is not perfectly sequential either.
Which means the honest way to think about colibri is as a memory-tiering system with a command-line interface. You are trading throughput for capacity. A 429 GB model does not fit in 32 GB of RAM. It does fit on a 1 TB drive. Whether that trade is worth it depends entirely on whether you need the model's capability more than you need its speed.
There is a second-order effect that matters for anyone doing agent work. Agents are bursty. They sit idle for minutes, then fire twenty tool calls. A local model that takes twenty seconds per response is unusable in that loop. A local model that takes two minutes is not usable either. So the honest use case is batch work: summarize this folder, classify these four hundred documents, draft these twelve product descriptions. Runs where you walk away.
The part I would test first
Three things are worth checking before you commit disk space.
Cold start versus warm. The 32-second figure in the README is for GLM-5.2 with a warm page cache, presumably. First run after boot will be slower, and on a spinning disk it will be much slower. Time it yourself.
Whether your model is actually supported. Thirteen engines is a lot by the standards of hand-written C, but it is a small slice of the open-weight ecosystem. The README notes that checkpoints sharing an architecture run unchanged, which is a useful escape hatch, and also a caveat: "same architecture" is doing quiet work in that sentence.
The license of whatever you load. colibri itself does not change the terms of the weights it runs. Qwen-Image-2.1 is research-only. Some of the language models are MIT. Some are not. The engine is neutral; the model is not.
Why this matters beyond one repo
For two years the story about local AI has been about quantization: get the model small enough to fit. Two bits per weight, 4 GB of VRAM, a phone. That line of work is real and it has produced genuinely useful things.
colibri points at a different axis. Instead of making the model smaller, make the memory hierarchy deeper. RAM is cheaper than VRAM. Disk is cheaper than RAM. The gap between "a model that exists" and "a model I can run" does not have to be closed by compression alone; it can be closed by accepting a slower tier.
That is not a new idea. It is how databases have worked for forty years, and how operating systems have worked for longer. What is new is that someone shipped it as a batch file for a 744B model, published the slow benchmarks alongside the fast ones, and included an image model in the box.
The repo will not replace your API subscription. It might replace the assumption that a 700B model is something you can only rent. And the picture it draws on eight CPU cores in two minutes forty is a decent reminder that "local" is not a binary, it is a budget you set.
Related articles
The Jensen Discount: What Nvidia Loses When Its Founder Walks Out
Nvidia lost roughly $600 billion in market value when its founder stepped down. The earnings did not change. The multiple did, and that says where the value really lived.
Chinese text in video always turns to mush. Dreamina handed the typesetting to code
Dreamina launched Motion Studio on November 10. It no longer generates frames; the model writes code that lays out text, shapes and pacing. Chinese titles stop blurring, but whether the content is right is still on you.
A community spreadsheet of taps that turns off the ads inside Chinese Android builds
A no-root tool that switches off the ads Chinese phone makers ship inside the system UI. Its method is a hand-written tap script per phone model, maintained by volunteers every time a ROM moves. It works, and the maintenance is the point.
GPT Image 2.6 held the token price and raised the bill
GPT Image 2.6 kept every token rate from 2.5 and still costs more per image, because more of the work now happens in output tokens. Eleven days into the ramp, the pricing structure says more than the leaderboard does.