JustVugg/colibri: Colibri: Run 744B MoE Models on Consumer
Run massive 744B MoE models on a 25GB RAM machine with pure C and zero dependencies.
★ 4.5 / 5 iKeep scoreScreenshots
Quick facts
Overview
Colibri is a lightweight, dependency-free C engine that enables running GLM-5.2 (744B MoE model) on consumer-grade hardware with as little as 25GB RAM. It streams experts from disk, keeping only essential parts in memory.
Getting started with GitHub - JustVugg/colibri: Run GLM-5.2 (744B MoE) on a 25GB-RAM consumer machine — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
- Clone repositoryClone the Colibri repository from GitHub using 'git clone https://github.com/JustVugg/colibri'.
- Build the engineCompile the C source code using a C compiler (e.g., gcc) as per the instructions in the repository.
- Download modelObtain the GLM-5.2 model files and place them in the specified directory as per the project documentation.
- Run inferenceExecute the compiled binary with appropriate parameters to start running inference on your local machine.
Pricing
| Plan | Price | Note |
|---|---|---|
| Open-source | Free | — |
💡 Indicative pricing — check the official rates at github.com · Updated 7/16/2026
Original pricing screenshotKey features
- Runs GLM-5.2 (744B MoE) on 25GB RAM machines
- Pure C implementation with zero dependencies
- Streams experts from disk with LRU caching
- Supports native MTP speculative decoding
- Includes MLA attention with compressed KV-cache
- Optional CUDA tier for pinned experts
Pros
- Extremely efficient memory usage for large models
- No dependencies - simple deployment
- Validated against transformers for accuracy
Cons
- Not byte-identical to non-speculative greedy in practice
- Requires significant disk space (~370GB for experts)
Use cases
- Running large language models on consumer hardware
- Edge AI applications with limited resources
- Research on efficient model inference
The verdict
Colibri is an impressive technical achievement that makes massive MoE models accessible on consumer hardware, though with some trade-offs in exact output consistency.
✓ Who should use it
AI researchers and engineers who need to run large MoE models without specialized hardware.
✕ Who should skip it
Those needing byte-exact reproducibility or who don't have sufficient disk space for the expert files.
Categories
Community
- GitHub Discussions ↗Official discussion forum for Colibri development
Great for
Featured in
Frequently asked questions
Is GitHub - JustVugg/colibri: Run GLM-5.2 (744B MoE) on a 25GB-RAM consumer machine — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦 free?
GitHub - JustVugg/colibri: Run GLM-5.2 (744B MoE) on a 25GB-RAM consumer machine — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦 is free to use.
What is GitHub - JustVugg/colibri: Run GLM-5.2 (744B MoE) on a 25GB-RAM consumer machine — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦 best for?
Developers and researchers needing to run massive models on limited hardware
Does GitHub - JustVugg/colibri: Run GLM-5.2 (744B MoE) on a 25GB-RAM consumer machine — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦 have an API?
No, GitHub - JustVugg/colibri: Run GLM-5.2 (744B MoE) on a 25GB-RAM consumer machine — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦 does not offer a public API.
Can I self-host GitHub - JustVugg/colibri: Run GLM-5.2 (744B MoE) on a 25GB-RAM consumer machine — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦?
Yes, GitHub - JustVugg/colibri: Run GLM-5.2 (744B MoE) on a 25GB-RAM consumer machine — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦 can be self-hosted.
Who should use GitHub - JustVugg/colibri: Run GLM-5.2 (744B MoE) on a 25GB-RAM consumer machine — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦?
AI researchers and engineers who need to run large MoE models without specialized hardware.
Reviews 4.0 ★ · 1 review
Colibri represents a significant breakthrough in efficient model inference, enabling researchers to run massive 744B MoE models on standard consumer hardware. The pure C implementation with zero dependencies makes it particularly attractive for deployment in resource-constrained environments. While there are some minor trade-offs in output consistency due to quantization effects, the ability to run such large models with modest RAM requirements is revolutionary. This tool shines for AI researchers working on edge deployment or those wanting to experiment with MoE models without access to specialized hardware.
No reviews yet. Be the first to review GitHub - JustVugg/colibri: Run GLM-5.2 (744B MoE) on a 25GB-RAM consumer machine — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦.
Write a review
Related tools
