Could soon allow small businesses to run advanced, private AI locally on existing hardware for free without cloud API fees.
What Is PrismML’s Bonsai 2 And What Does It Do?
PrismML is a Caltech-founded AI startup that compresses large language models until they run on standard PCs and smartphones. Its latest release, Bonsai 2 27B, compresses Qwen3.8 27B, a widely used open source model from Alibaba, down to 5.9 GB.
The technique is ternary weights. A normal model stores each weight in 16 bits, and PrismML’s approach simplifies that to 3 values, +1, -1, or 0, which the company pairs with group-wise scaling for 1.76 effective bits per weight, per its official release notes.
The company raised a $22.25 million seed round from Khosla Ventures, Cerberus Capital and Caltech, and counts Databricks co-founder Ion Stoica as an adviser. CEO Babak Hassibi is a Caltech professor who specializes in compression technologies.
PrismML’s bet is that capable reasoning models do not have to be large, and the download counts suggest the market agrees.
How Many People Actually Use PrismML’s Models?
Adoption is the loudest data point in the signal. The original Bonsai has been downloaded over 11 million times, and PrismML’s smaller variants have pulled another 2.6 million, according to TechCrunch’s reporting.
Bonsai 2 landed on Hugging Face with a 262K-token context window, multimodal text-and-image input, and an Apache 2.0 license. A business can ship products on it without a commercial negotiation.
The honest caveat is that those download counts come from the company itself. No third-party audit of the figures exists yet, and the model is 1 day old.
Momentum is real, vendor-reported, and pointed in one direction.
Is Bonsai 2 Good Enough To Replace A Cloud API?
For routine work, the benchmark math says yes. Bonsai 2 retains 98.2% of Qwen3.8 27B’s aggregate benchmark performance, up from 95% for the first Bonsai in the series.
Hassibi concedes compression will always cost something, and whether 100% parity is reachable stays an open question. The practical read in the coverage is that a 2% degradation rarely shows up in actual use, because benchmarks already reflect real tasks imperfectly.
Multiverse Computing is working the same problem from Spain, which makes local compression a category rather than one company’s stunt. Rumors of talks with Apple add a ceiling nobody at the company will confirm.
Drafting, summarizing and internal search clear the bar today, and frontier reasoning stays on the API.
What Breaks When You Run Bonsai 2 In Production?
The 2% gap compounds in the wrong places. Coding agents, tool use and long-horizon agentic tasks are exactly where small errors stack across steps, and that is where PrismML claims its strongest retention.
The company’s demos ran coding agents and computer-use workflows on an NVIDIA GeForce RTX 5090, which is a gaming card, not a server. Hardware still gates the experience: 5.9 GB fits on a standard PC and possibly a high-end smartphone.
The next release targets models in the several-hundred-billion-parameter range, and Hassibi expects larger models to compress more cleanly because there is more room to shrink them without losing capability. Check what your routine queries cost today on our live AI API pricing tracker, because that is the bill Bonsai 2 is trying to zero out.
Run it for routine work now, and benchmark it against your real tasks before you trust it with a workflow that compounds.
The copier lease in a 12-person shop bills by the click, and the meter runs on every page whether that page sells anything or not. By year end the ledger holds a number big enough to buy the machine outright, twice over. The click charge was the no-commitment option, never the cheap one.
Bonsai 2 is the owned machine: 5.9 GB on a workstation you already paid for, 98.2% of the benchmark scores, and a query cost of nothing. The 27B-class model that used to need a meter and a contract now sits beside the spreadsheets on the same drive.
Run the math the way you would on any metered supplier: pull a month of API spend, hold it against the work those queries did, and see which jobs were rentals all along.
Routine, sensitive work was always the first thing that should have come home.
Should You Run Bonsai 2 Instead Of Paying For An API?
The decision splits on sensitivity and volume. Work that touches client data or internal financials and repeats daily is the first thing to move, because it costs nothing per query and the data stays on hardware you own.
Cloud APIs still win where frontier reasoning, elastic scale or uptime guarantees matter. Stoica’s framing in the coverage is that intelligence on a device you already bought is free and private because nothing gets sent to the cloud.
The audit takes one afternoon: pull a month of API invoices, sort the calls by sensitivity, and price the local alternative at $0 per query.
Audit the API bill this week, move routine sensitive work local, and keep the cloud budget for tasks that need frontier reasoning.
Source: TechCrunch AI
