An open-source inference engine now runs frontier-scale open-weight models, some as large as 2.8 trillion parameters, on ordinary desktop or workstation hardware instead of a hosted, per-token API.
A team running a steady, high-volume AI workload, an overnight document summarization batch, a support-ticket triage queue, an internal research pipeline, usually ends up paying a hosted API by the token. Together.ai's public pricing lists GLM-5.2, a 744-billion-parameter open-weight model, at $1.40 per million input tokens and $4.40 per million output tokens, with a lower cached-input rate of $0.26 (Together.ai pricing). Colibri, an open-source inference engine that reached its v1.11.0 release this week, runs that same model, and eight others up to 2.8 trillion parameters, on a desktop or workstation a business already owns, no rack of GPUs required.
Colibri is a single C program with no Python runtime dependency and no required GPU. It treats a computer's VRAM, RAM, and disk as one hierarchy for a model's weights rather than requiring the whole model to fit in fast memory. The trick behind that: a mixture-of-experts model like GLM-5.2 only activates a fraction of its parameters on any given token, roughly 40 billion of its 744 billion, and only about 11 gigabytes of that changes from one token to the next. Colibri keeps the small, always-used part of the model resident in RAM and streams the rest of the routed experts from disk on demand, tracking which experts a workload calls on most often and keeping those pinned in faster memory over time.
The project shipped v1.11.0 on September 13, adding a ninth supported model family and closing five bugs across four of its engines, on top of 56 pull requests since the prior release. The engine itself is Apache 2.0 licensed, and the reference GLM-5.2 weights are released separately by Z.ai under MIT, so a business can adopt both without a non-commercial restriction working against it. The repository has picked up roughly 2,200 stars in the past day on top of 31.8k total, and lists 141 contributors and 17 tagged releases, a reasonable sign that it is actively maintained rather than a weekend experiment.
For a business, the shift is about where the cost lands. A hosted API bills every time a token moves in or out, which adds up for work that runs constantly rather than occasionally. Colibri trades that recurring bill for a one-time hardware cost and some ongoing maintenance. The project's own hardware guidance shows GLM-5.2 running with no GPU at all, on 16 gigabytes of RAM minimum and about 372 gigabytes of disk for the model weights, and its published benchmark log records roughly 1.8 tokens per second on a 128-gigabyte CPU-only desktop. That is far slower than a hosted API's replies, but for work that does not need to happen in real time, batch processing overnight, internal document indexing, offline research on sensitive material that should not leave the building, the marginal cost per token approaches zero once the hardware is paid for.
This fits a solo founder or an internal IT or engineering team that already has, or is willing to buy, a workstation with enough RAM and disk, and that runs enough token volume for the math to matter. It is a poor fit for a team that needs a live, customer-facing response in under a second, or one without anyone comfortable at a command line.
The setup cost is real and worth naming honestly. Getting a model running means downloading several hundred gigabytes of converted weights, the GLM-5.2 container alone is about 372 gigabytes, and the largest supported model, Kimi K3, needs roughly 1.6 terabytes of disk and 32 gigabytes or more of RAM. The project's own documentation warns that using the wrong converted checkpoint has caused earlier versions of these models to loop or never stop generating, so getting the right container matters. Colibri describes itself as a research platform first and an inference engine second, and says plainly that there is no guaranteed speed, only measured results on whatever hardware is on hand.
Colibri does not make a trillion-parameter model fast to run. It makes it possible to run one at all without renting it by the token, which is a different kind of decision than most businesses are used to making about AI. The real question is not whether the tokens are free, they are not once hardware and setup time are counted, but whether a given workload is patient enough for that tradeoff to pay off.