How we approached this review: We based this review on the project’s official documentation, published benchmarks, community hardware results, and our experience evaluating local AI software and hardware. While we have not yet built a dedicated Colibrì test system, we’ve clearly distinguished measured results from project estimates throughout this review.

For years, running a model with 744 billion parameters meant renting expensive cloud infrastructure or buying enterprise-grade hardware. That assumption no longer holds true.

Colibrì, an open-source AI inference engine developed by Italian programmer JustVugg, can run the full GLM-5.2 744B model on everyday hardware—even without a dedicated GPU. It’s one of the most talked-about AI projects of the year because it challenges a long-standing belief: that state-of-the-art AI models belong only in data centers.

There’s one important detail that rarely makes the headlines.

The software is free, but achieving a practical experience requires capable hardware. You’ll need around 380 GB of free storage, 16–24 GB of RAM, and hardware that can keep up with constant disk streaming. Without the right setup, performance can range from surprisingly usable to frustratingly slow. Understanding what your hardware can realistically handle before downloading can save you hours of frustration—and unnecessary upgrades.

With more than 20 years of experience in hardware and application research and development, we evaluate technology based on what matters after the marketing ends—real-world performance, reliability, upgrade potential, and long-term value. Every recommendation we make comes from detailed research, component-level analysis, practical usability testing, and years of industry experience.

Whether you’re an AI developer, machine learning engineer, software developer, computer science student, researcher, home lab enthusiast, content creator, or simply someone interested in running large language models (LLMs) locally, this guide is written with your needs in mind. We’ll separate the impressive claims from the practical reality, explain the hardware requirements in plain English, and help you decide whether Colibrì is worth installing on your system.

By the end, you’ll understand how Colibrì works, why it’s attracting so much attention in the AI community, what hardware delivers the best experience, and whether running GLM-5.2 on your own hardware makes sense for your budget.

Why a model this size normally needs a datacenter

Three major obstacles stand between a model of this scale and an ordinary computer.

• The memory wall: Conventional inference loads the entire model into fast memory before generating anything. At 4-bit precision, a 744B model is still hundreds of gigabytes. Even aggressively compressed community builds of GLM-5.2 need around 239 GB of combined RAM and VRAM — more than most workstations have, let alone laptops.
• The VRAM wall: GPUs are fast but memory-poor. A consumer flagship gives you 24–32 GB. Stacking cards means a motherboard with the lanes, a power supply that can feed them, and a budget that starts around the price of a used car.
• The setup wall: CUDA versions, Python environments, quantization formats, dependency conflicts. Plenty of people give up here before hardware ever becomes the limit.

Colibrì challenges the biggest assumption behind running large AI models. Instead of forcing the entire model into memory, it places different parts where they’ll be used most efficiently.

How it works, without the jargon

Imagine walking into one of the world’s largest libraries. If you wanted the answer to one question, would you carry every single book to your desk first? Of course not. You’d bring only the few books that contain the information you need and leave the rest on the shelves.

Colibrì applies the same idea.

Instead of loading the full model into memory, Colibrì keeps only the parts it needs most readily available. The rest remains on your SSD until it’s needed. As each new token is generated by GLM-5.2, Colibrì retrieves only the relevant experts from storage. This approach dramatically reduces the amount of RAM needed to run such an enormous model.

This works because of how GLM-5.2 is designed.

Unlike traditional AI models that rely on one giant neural network for every task, GLM-5.2 uses a

Mixture-of-Experts (MoE) architecture. Think of it as having thousands of specialists instead of one general expert. When GLM-5.2 predicts the next word, it doesn’t consult every specialist. It asks only the few that are best suited for that specific task.

Out of its 744 billion parameters, only about 40 billion parameters—roughly 5.4% of the entire model—are active for each generated token. The remaining experts simply wait until they’re needed. Colibrì takes advantage of this behavior by loading only those active experts from storage, instead of forcing your computer to keep the entire model in memory.

That’s the key idea behind Colibrì. Rather than making your computer carry the weight of the whole model at once, it delivers only the pieces needed at the exact moment they’re required. It’s a simple change in strategy that makes running this frontier model on your own computer possible.

So Colibrì splits it in two:

The project’s own analogy is sharper than the library one, and worth borrowing if you’re technically inclined: it’s a JIT compiler, but for weights. A JIT never compiles an entire program up front — it watches what actually runs and compiles the hot paths just in time. Colibrì makes the same bet about a 744-billion-parameter space. Parameters aren’t state to be held; they’re data to be staged, exactly when the router proves they’re needed.

Several refinements make this practical rather than merely clever:

• A learning The engine learns which experts your workload uses most often and keeps those in RAM.
• Router lookahead. A prefetch thread runs a layer ahead of the computation. Routing turns out to be 6% predictable one layer in advance, so most disk reads can be started before they’re needed.
• Compressed conversation The attention mechanism stores 576 floats per token instead of 32,768 — 57× smaller — and persists it between sessions, so reopening a conversation doesn’t re-read everything.

The reality check before you spend anything

Here is where most coverage of this project stops short, and where a buying decision actually gets made.

Colibrì is free. Running it is not.

You need roughly 380 GB of free disk space, and not just any disk. Because the experts stream from storage on every single token, your drive’s read bandwidth is your token speed. This is not a minor tuning detail — it is the dominant variable in the entire system. A cold token requires around 11 GB of disk reads.

That number is worth pausing on, because it reframes everything. Generating one word can mean pulling 11 gigabytes off your drive. On a slow disk that’s a wait measured in tens of seconds. On a fast NVMe it’s a second or two.

The second constraint is RAM, and it bites in a way that surprises people. On a machine with about 24 GB, the engine auto-caps its expert cache to two slots per layer — which means on small-RAM systems the RAM cap, not the disk, becomes the binding constraint. You can put the world’s fastest SSD in a 16 GB laptop and still be slow, because there’s nowhere to keep what you fetch.

Minimum viable, per the project’s documentation:

What the benchmark data actually proves

This may be the most valuable section of the project’s documentation.

One contributor ran the cleanest possible experiment: same CPU, same RAM, same learned cache history — only the SSD swapped.

5.8× the disk bandwidth bought 2.9× the tokens, and — more importantly — moved the bottleneck off the drive entirely. That’s the single most actionable data point in the whole project. It also tells you where spending more money stops delivering meaningful gains.

The full spread across community-reported machines:

Two patterns worth internalising. First, RAM capacity eventually beats disk speed — the EPYC box with 430 GB of RAM cached 98% of experts and stopped touching the disk at all. Second, there’s a genuine ceiling on single-machine systems: even excellent single-machine setups land between 1 and 2 tok/s. That’s reading pace, not chat pace.

Build tiers: what to buy, in what order

We’ve grouped these by what the measurements justify, not by price bracket. Skip any tier where you already meet the requirement.

Tier 0 — Storage: buy this first, always

Before upgrading anything else, make sure you have roughly 380 GB of fast NVMe storage available.

The sweet spot — PCIe 4.0, 2 TB

For most people this is the right answer. PCIe 4.0 drives deliver 5–7 GB/s, which is enough to move the bottleneck off your disk on a typical desktop, at roughly half the cost per terabyte of the newest generation. The WD Black SN850X 2TB appears in the community results powering a Framework 13 to 0.37 tok/s, and the Samsung 990 PRO 2TB is its closest equivalent.

If your motherboard supports PCIe 5.0, this is the upgrade worth considering.

If your motherboard has a Gen 5 slot, the Samsung 9100 PRO 4TB is the drive from the A/B test above, measured at 8.81–11.48 GB/s in this exact workload. The 4 TB capacity also leaves room for the model plus a second copy (see dual-drive below). This is the highest-confidence recommendation in the article because we can point at a controlled experiment on the same machine.

For laptops that can’t take a second drive — an external enclosure over USB4/Thunderbolt gets you there without opening anything. A USB4 NVMe enclosure paired with a bare drive typically lands around 3–3.5 GB/s. Slower than internal, but workable, and it keeps 372 GB off your system drive.

Capacity guidance: 1 TB is genuinely tight once you account for the OS and the fact that a 380 GB file on a nearly-full SSD will also slow down. 2 TB is the practical minimum; 4 TB if you plan to keep more than one model.

A note on drive health. Colibrì’s streaming is read-only, and reads don’t meaningfully wear an SSD, so you’re not burning through endurance. The two things to actually watch are sustained thermals — hours at full read duty will heat a cheap drive — and swap traffic if you under-provision RAM.

A heatsink or a motherboard with M.2 cooling is a reasonable few dollars on a drive that’ll run hot for hours.

Tier 1 — RAM: the second lever, and eventually the bigger one

If you’re at 16–24 GB, this should be your next upgrade, and it may matter more than the SSD.

At 24 GB the engine restricts itself to two cached experts per layer and stays effectively cold no matter how fast your drive is. Getting to 64 GB lets the learning cache pin a meaningful hot set. Getting to 128 GB is where the community numbers start clustering above 1 tok/s.

• Desktop, DDR5: a 64 GB (2×32) DDR5 kit is the value pick; 128 GB (4×32) if your board and budget allow.
• Desktop, DDR4: still perfectly viable here — this is a bandwidth-and-capacity problem, not a latency one. 128 GB DDR4 kits are inexpensive on the used and OEM market.
• Laptop: check whether yours has SO-DIMM slots before anything Many thin machines are soldered, which caps your ceiling permanently. If it’s upgradeable, a 64 GB SO-DIMM kit is the single best thing you can do to it.

Tier 2 — Dual-drive streaming (the clever, cheap trick)

This one is under-appreciated. Colibrì supports streaming from two SSDs at once: put a copy of its weights on each, and the engine hashes each expert to one drive, weighted by measured bandwidth. Aggregate bandwidth is the sum of both.

A 9 GB/s + 3 GB/s pairing reads roughly 33% faster than the fast drive alone. And because a partial mirror is valid, a smaller second drive still helps — it doesn’t need to hold the whole model.

So if you already own a decent NVMe, adding a second one is often better value than replacing it with something faster. A second 2 TB PCIe 4.0 drive in a spare M.2 slot is a cheap, measurable gain.

Tier 3 — Whole-machine options

If you’re buying a system rather than upgrading one, the data points to some unconventional answers.

Apple Silicon with lots of unified memory performs unusually well here — the M5 Max with 128 GB reached 2.06 tok/s using the Metal backend, among the best single-machine results reported.

Unified memory means the “RAM cache” and “GPU memory” are the same pool, which suits this workload. A Mac Studio or MacBook Pro with 128 GB is a legitimately strong option, if an expensive one. Note the Metal backend is still marked experimental.

Used server hardware is the value play. The most instructive result in the entire dataset is the EPYC 7443 with 430 GB of RAM hitting a 98% cache hit rate — it simply stopped using the disk. Older EPYC and Xeon platforms with large registered-DIMM capacity are startlingly cheap secondhand.

A used EPYC or Xeon workstation with 256 GB+ will outperform a far pricier gaming PC at this specific task. It’ll be loud and power-hungry; it will also be several times cheaper than the Apple route.

Mini PCs with unified-ish memory — the Ryzen AI Max+ 395 platform (as in the Framework Desktop) posted 1.10–1.83 tok/s with 128 GB of LPDDR5x. A Ryzen AI Max mini PC is a quiet, compact middle path between a laptop and a server.

Tier 4 — GPUs: mostly skip this

Here’s the conclusion that surprised us most.

Colibrì now ships CUDA, AMD ROCm and Apple Metal backends, so a GPU can hold a tier of experts in VRAM. But the documentation contains a striking measurement: with proper thread tuning, an AVX-512 CPU matched an RTX 5090 on expert matrix multiplication. In one contributor’s setup the CUDA expert tier contributed approximately 0%.

The conclusion the project draws — and we agree — is that the GPU tier only earns its VRAM when the CPU is the weak link. For most builds, money spent on a graphics card would deliver more tok/s if spent on RAM or a faster SSD instead.

The exception is the extreme end: with six RTX 5090s and full expert residency you reach 6.8 tok/s. That’s a genuinely different experience, at a genuinely absurd price.

What not to buy

• QLC / DRAM-less SSDs. The Crucial P3 result above is the cautionary tale — 1.51 GB/s and a machine spending two-thirds of its time Cheap high-capacity drives are exactly wrong for this workload. Check for TLC NAND and a DRAM cache.
• External USB x drives. At roughly 1 GB/s you’ll be measuring seconds per token. USB4/Thunderbolt or internal only.
• Spinning hard Non-viable. Random-read latency makes this unusable regardless of capacity.
• A GPU as your first See above. Storage, then RAM, then reconsider.

How Colibrì compares to the usual local-AI tools

It’s worth being clear here: for most people, most of the time, Colibrì is not the right tool. It solves a specific problem.

The distinction that matters: Ollama and llama.cpp are capped by your memory. Colibrì is capped by your storage. That’s a fundamentally different budget, and it’s why a 372 GB model becomes reachable.

If your goal is a fast, private assistant for everyday work, install Ollama and run a modern 4B–8B open model instead. It’ll be 100× faster and perfectly capable. Choose Colibrì when you specifically need this class of model — frontier-scale reasoning and coding — running on hardware you own.

Does the compression hurt the model?

To the project’s credit, it measures this instead of asserting it.

The int4 container scored 62.5% mean accuracy across HellaSwag, ARC and MMLU. A controlled comparison of full precision against int4 on a smaller model put the pure quantization cost at −8.2 percentage points, concentrated on the hardest questions — where compressing weights erodes the fine margins difficult reasoning depends on.

Crucially, the packaging matters as much as the precision. Group-scaled quantization recovers roughly 63% of that loss versus the older per-row approach. This has a direct practical consequence:

Download the right model file. Use the group-scaled (gs64) container with the int8 prediction head.
The older per-row mirrors measure about 9 points worse and are the documented cause of generations that loop and never terminate. If you followed early coverage of this project, you may have been pointed at a deprecated file — check before you spend hours downloading 372 GB.

Who this is genuinely for

Worth your time if you:

• Want frontier-class model quality with complete privacy — legal, medical or confidential material that cannot leave your machine
• Have batch work where latency is irrelevant: overnight document analysis, corpus summarisation, bulk drafting
• Are evaluating whether to commit to an API contract and want to test against a frontier open model at zero marginal cost
• Enjoy understanding how large AI models work, and want to watch a 744B model think on your own hardware

Not worth it if you:

• Want an interactive Even good setups land around 1–2 tok/s. A short reply takes minutes.
• Don’t have ~380 GB to spare, and don’t want to buy it
• Need to serve multiple users — it handles one sequence at a time
• Just want a capable private A quantized 8B model in Ollama is a better daily driver by every practical measure.

Frequently asked questions

What is Colibrì?

An open-source inference engine, Apache-2.0 licensed, by Italian developer JustVugg. It runs very large Mixture-of-Experts models on ordinary hardware by keeping 9.9 GB resident in RAM and streaming 19,456 routed experts from disk on demand. It surpassed 20,000 GitHub stars within a month of release.

Do you need a GPU?

No. It runs CPU-only by default, which is the entire point. CUDA, AMD ROCm and Apple Metal backends exist, but measurements show a well-tuned AVX-512 CPU can match an RTX 5090 on this workload — so a GPU is usually not the upgrade that pays.

How much storage do I actually need?

About 380 GB free for the int4 model, and it should be an NVMe SSD. Streaming speed directly determines token speed. Plan for a 2 TB drive minimum once you account for your OS and headroom.

How much RAM?

16 GB is the floor and 24 GB+ is recommended, but the honest answer is that more RAM keeps paying. At 24 GB the engine caps its cache and stays cold; at 128 GB community results cluster above 1 tok/s; at 430 GB one machine cached 98% of experts and stopped using the disk entirely.

How fast is it, realistically?

On a typical well-specified desktop, somewhere between 0.3 and 2 tok/s. Fine for batch work, too slow for conversation. Multi-GPU workstations reach 5.8–6.8 tok/s.

Which model file should I download?

The group-scaled (gs64) int4 container with the int8 prediction head, roughly 372 GB. Avoid the older per-row mirrors — they measure about 9 points worse on quality and cause non-terminating generations.

Does it work on Windows and Mac?

Yes, both natively — Windows 10/11 without WSL, and macOS with an experimental Metal backend. Prebuilt binaries are published for Linux, Windows and macOS, so no compiler is needed. Python 3 is required for the launcher and API gateway; the engine itself is pure C.

Is it only for GLM-5.2?

No. OLMoE runs today, and the approach is architecture-agnostic — any MoE model with routed experts can be staged the same way. Support for Kimi K2, Qwen3 MoE and MiniMax is on the roadmap. That’s the part with long-term significance: this is a technique, not a one-model trick.

The verdict

Colibrì is one of the most impressive breakthroughs we’ve seen in on-device AI this year.

Its core idea is surprisingly practical. Instead of forcing your computer to load the entire model into memory, it uses the hardware you already own more intelligently. By keeping only the necessary parts of the model in memory and streaming the rest from storage, Colibrì makes running one of the largest open language models available today possible without expensive AI servers.

Equally important, the project is unusually transparent about its limitations. The developers don’t promise miracle performance. They openly state that Colibrì isn’t fast, and every published benchmark is presented as a real measurement rather than a marketing claim. That level of honesty is uncommon and gives the project far more credibility than exaggerated performance numbers ever could.

Even so, it’s worth setting realistic expectations before investing in new hardware.

While the software is free, building a system that runs it well still requires a meaningful hardware investment. Your experience will depend far more on your storage speed and available RAM than on having the latest graphics card. A fast NVMe SSD and sufficient memory can transform the experience, while slower storage quickly becomes the biggest bottleneck.

If you’re planning to build or upgrade a system specifically for Colibrì, this is the order we recommend:

1. A fast NVMe SSD with 2 TB+ — the Samsung 9100 PRO if you have PCIe 0, the WD Black SN850X if you don’t. Without a fast NVMe SSD, performance drops to the point where Colibrì becomes impractical for most users.
2. RAM to 64 GB, then 128 GB — DDR5 or DDR4, whichever your board This is where the second-biggest gains hide.
3. A second SSD for dual-drive streaming — any decent 2 TB NVMe adds roughly a third more read bandwidth for modest money.
4. Reconsider the On this workload, it’s usually the worst dollar-per-token upgrade available.

If your goal is simply to have a responsive, private AI assistant on your computer, there’s an easier

path. Installing Ollama and running a modern 8B open-weight model will provide a faster, smoother experience on most consumer PCs. Colibrì isn’t designed to replace that workflow.

More importantly, Colibrì shows that even the world’s largest open AI models can now run outside traditional data centers. That’s a meaningful shift for developers, researchers, and anyone who prefers AI running on their own machine and values privacy, ownership, and complete control

over their hardware.

Hardware prices, availability, and benchmark results will continue to change as Colibrì evolves. Always verify the latest documentation and current pricing before making purchasing decisions. As the software continues to receive frequent updates, real-world performance is expected to improve.

***Disclaimer***

This blog post reflects our research, analysis, and opinions based on available product information, user feedback, and industry knowledge. It should not be taken as the official position of any brand, manufacturer, or company mentioned here. We make every effort to keep this guide accurate, but product specifications, pricing, and availability may change after publication. We recommend double-checking important details before making a purchase.

Some links in this article may be affiliate links. If you choose to buy through these links, we may earn a small commission at no extra cost to you. This helps support our work and allows us to keep publishing in-depth, unbiased reviews. Affiliate partnerships never influence our recommendations.

Opinions expressed by readers are their own and do not necessarily reflect ours. We are not responsible for outcomes resulting from the use of information on this site. Please seek professional advice where appropriate.

All product names, logos, and brands mentioned are the property of their respective owners. These names are used for identification and informational purposes only and do not imply endorsement.