Bonsai 27B Runs on an iPhone 17 Pro Max at 11 Tokens a Second

ByGeoff Dyers
bonsai 27b iphone 17 pro max: An iPhone running an on-device AI model, illustrating phone-class local inference.

PrismML announced Bonsai 27B on July 14, 2026 as a 1-bit version of Qwen3.6 27B that it says fits and runs natively on an iPhone 17 Pro Max at about 3.9 GB and roughly 11 tokens per second. The company’s model card says the phone result uses MLX on iPhone 17 Pro Max, which makes this a specific hardware claim, not a blanket “runs on phones” result.

PrismML is a startup focused on compressed local models, and Bonsai 27B is its new family of binary and ternary builds derived from Qwen3.6 27B. The release matters because a 27B-class model is far larger than the kind of on-device model Apple itself has publicly described: Apple’s 2025 technical report says its on-device AFM 3 Core is a 3B-parameter model.

Bonsai 27B’s reported phone-class footprint and throughput

The headline numbers are simple. PrismML’s Hugging Face model card lists the 1-bit mlx build at 3.9 GB, with peak memory of 4.37 GB for a 512-token prompt and 10.9 tokens per second on iPhone 17 Pro Max. That is the clearest available answer to whether Bonsai 27B can really run on a phone: PrismML says yes, on one specific phone, under one specific runtime stack.

The memory budget is the whole trick. A conventional 27B model in ordinary low-bit form usually lands in laptop territory, not phone territory, which is why this release immediately joins the broader debate over local LLMs versus ChatGPT trade-offs. PrismML’s pitch is that reducing transformer weights to 1 bit cuts the model enough to fit inside a phone-class envelope without dropping all the way down to a tiny assistant model.

The phone claim is currently sourced mainly to PrismML’s release, homepage, and model card, not to a third-party benchmark lab. Independent coverage at 9to5Mac and MacRumors’ write-up of The Information’s reporting repeats the Apple-device angle, but the core performance numbers still come from PrismML.

How Bonsai 27B compares with conventional Qwen3.6 27B builds

PrismML’s technical abstract says Bonsai 27B keeps full 27B-class transformer width while compressing weights into binary and ternary formats. That matters because the comparison is not against a smaller dense model; it is against the same base architecture shrunk aggressively enough to run locally.

The company says its ternary build averages 80.49 on its evaluation suite, while the 1-bit phone-oriented build averages 76.11. The 1-bit version gives up benchmark score to hit the smaller memory target. That trade-off is explicit in the release: the stronger retained-capability figures belong to Bonsai variants compared against PrismML’s Qwen3.6 27B base on the company’s own benchmark suite and methodology.

A compact way to read the lineup is this:

Build Reported result
1-bit Bonsai 27B mlx 3.9 GB, 10.9 tok/s on iPhone 17 Pro Max, 76.11 average benchmark score
Ternary Bonsai 27B 80.49 average benchmark score
Apple AFM 3 Core 3B parameters, on-device model baseline

That table does not prove Bonsai 27B beats cloud flagships. It does show why this release is more than a curiosity. A 27B-class model living inside a roughly 4 GB runtime envelope is a very different proposition from the usual “small local model, big quality drop” story.

Long context is where the fine print arrives. PrismML’s model card says peak memory rises sharply with longer context unless 4-bit KV cache compression is enabled. In practice, that means the phone result is easiest to reproduce for shorter prompts and sessions, not for giant context windows.

For developers, the interesting part is not just the weights but the stack. PrismML ships this as an MLX build, so the result lives inside Apple’s own silicon-and-runtime environment rather than a generic cross-platform mobile path. That fits the recent push to build a workable on-device AI app stack around runtime-specific packaging, memory management, and local inference tooling.

The 1-bit version gives up benchmark score to hit the smaller memory target.

Why the release matters for the local-vs-flagship trade-off

Bonsai 27B does not settle the case for local AI over flagship cloud AI. It does sharpen it. The release pushes the “good enough locally” line upward: instead of asking whether a 3B-ish assistant can do lightweight tasks on-device, PrismML is arguing that a compressed 27B model can cover more practical work while staying private, offline-capable, and app-embedded.

That is still different from saying local now beats the best hosted systems. PrismML’s own numbers are about retention versus its Qwen3.6 27B base, not head-to-head wins over frontier cloud models. If your job needs the widest tool use, the deepest reasoning, or huge context windows, the best cloud systems still have the easier path. But for the growing set of cases where developers mainly care about latency, privacy, predictable cost, and shipping offline features, Bonsai 27B makes the local side look less like a toy.

The release also clarifies the hardware question. The most credible near-term path for strong local models is not “any phone,” but premium devices with enough unified memory, tight runtimes, and aggressive compression. That makes this as much a story about local LLM stack choices as about raw model quality.

PrismML’s announcement says Bonsai 27B is available now under the Qwen Research License. The main public artifact for the phone build is the Hugging Face model card, which is where PrismML has posted the runtime and benchmark details.

Key Takeaways

Further Reading