Gemma 4’s 2 GB Mac Story Is Really SSD Streaming

ByGeoff Dyers
gemma 4 26b 2 gb ram mac: Apple Silicon MacBook on a desk for local AI model work

Gemma 4 26B-A4B does not, on the best citable evidence here, run as a true all-in-RAM 2 GB model on Macs. The strongest published figures from the project behind the claim put Gemma-only resident memory at about 3 GB with SSD-streamed expert loading, compared with about 6 GB using memory-mapped llama.cpp workflows and about 13 GB for vanilla 4-bit MLX on Apple Silicon, according to the project’s Hugging Face page and an independent setup guide.

That makes the real story narrower, but still interesting. The project is showing disk-backed Mixture-of-Experts inference on Macs, not a full end-to-end local workflow with the whole model sitting in 2 GB of memory. The project’s own published setup guidance also asks for 16 GB or more of RAM and about 30 GB of free disk, which is a more grounded statement than “any M-series Mac.”

Gemma 4 26B-A4B is one of Google’s Mixture-of-Experts models. The Gemma 4 technical report describes Gemma 4 as spanning MoE architectures from 2.3B to 31B parameters, and an MLX Swift issue discussing Gemma 4 support describes the 26B variant as having about 3.8 billion active parameters per token. That “26B” label is about total model size, not the amount of compute or memory touched at once. In MoE systems, only some experts fire for each token. That is what makes this kind of trick possible.

What the 2 GB claim is actually measuring

The closest citable documentation from the project does not publish a plain “2 GB full workflow” number. Its Hugging Face deployment page says Gemma 4 26B 4-bit plus Sniper uses about 3 GB resident RAM, while Gemma plus Falcon vision uses about 5 GB resident RAM. The same page contrasts that with vanilla MLX at about 13 GB for Gemma 4 26B 4-bit.

That is a resident-memory claim, not a claim that the whole model file has shrunk to 2 or 3 GB. The same page recommends about 30 GB of free disk space, and the project’s GitHub repository says it works by flash streaming experts on Apple Silicon rather than pinning the full model in memory.

“Only the hot experts plus a small DRAM cache stay resident; cold experts are streamed from flash on demand.”, Walter Grace, mac-code repository

That is the key distinction. Low resident RAM here means the runtime is keeping a small working set in memory while using the SSD as an extension of the model store. It does not mean a Mac is somehow running a 10 GB to 16 GB class model file as a literal 2 GB in-memory artifact.

A quick file-size reality check makes that plain. The Unsloth GGUF listing includes quantized Gemma 4 26B-A4B files such as UD-IQ2_M at 9.97 GB and MXFP4_MOE at 16.6 GB, while a Bartowski IQ4_XS GGUF page lists a 14.2 GB file used in llama.cpp testing. The storage footprint never disappeared. It moved off resident memory and onto disk-backed access.

How Gemma 4 26B-A4B normally fits on Macs

Against standard local setups, the SSD-streamed result is real. It is just not magic.

The MLX Expert Sniper page lays out the cleanest side-by-side numbers in this source set:

Setup Published memory figure
Sniper + Gemma 4 26B 4-bit about 3 GB resident RAM
llama.cpp with mmap about 6 GB RAM
Vanilla 4-bit MLX about 13 GB RAM

An independent Unsloth setup guide reports the same middle ground from a different angle: Gemma 4 26B MoE can run at about 6 GB RAM with memory-mapped files on Apple Silicon. Another independent data point is higher: an issue in mlx-swift-lm says the downstream 4-bit MLX model required about 26 GB of RAM in that context. Different runtimes and packaging choices matter a lot.

That spread is the useful answer for Mac users. “Gemma 4 26B on a Mac” is not one memory number; it is a stack choice. If you are comparing local LLM stack choices, the question is not just “can it load,” but how it loads: full MLX residency, mmap-heavy access, or expert streaming from SSD.

The project’s own numbers also narrow the “any M-series Mac” framing. The Hugging Face page recommends 16 GB+ unified memory, and the GitHub repository reports measured performance on specific machines, especially a 16 GB Mac mini M4. That is a useful demo target. It is not the same as a broad hardware guarantee across the entire M-series line.

The storage footprint never disappeared. It moved off resident memory and onto disk-backed access.

The speed and quality tradeoffs of SSD-streamed MoE inference

The upside is obvious: resident memory drops hard enough to make a 26B-class MoE model practical on Macs that would struggle with a more conventional load path. The downside is just as obvious: you are paying for that with storage traffic, caching behavior, and workflow constraints.

The mac-code repository describes an architecture that predicts and prefetches experts so the SSD is not hit blindly for every token. That matters because random storage fetches are slower than RAM by a wide margin. The trick is to behave less like a naive pager and more like a tiny specialist librarian: keep the experts you are likely to need next close at hand, and fetch the cold ones before the model stalls.

The project publishes speed figures, but they are benchmarks from particular Macs, not universal promises. Its GitHub page includes measured results on Apple Silicon and presents the method as a way to stay usable despite the streaming overhead. That is plausible. It also means readers should resist the usual local-AI demo error: treating one tightly scoped benchmark as a general hardware law. That is how AI claim verification failures happen in the first place.

Quality is a separate axis. The Gemma 4 technical report and the broader Gemma 4 developer shift story are about model design and capability, not this particular Mac runtime. Expert streaming changes deployment economics, not the underlying model weights. In practice, though, the workflow can still feel different if slower token generation, cache misses, or storage pressure make longer contexts and interactive use less smooth than a more RAM-heavy setup.

One more tradeoff sits in the background: disk wear and disk dependency. The method depends on SSD access as part of normal inference, so free space, storage speed, and sustained I/O matter more than they do in a simple all-in-memory run. The Hugging Face page explicitly asks for about 30 GB of free disk, which is the kind of requirement that gets lost when a demo collapses everything into one RAM number.

So is “2 GB RAM” meaningful? Only in the narrow sense of resident-memory measurement, and even then the best citable figure here is closer to 3 GB for Gemma-only than 2 GB. As a real-world Mac buying or setup claim, it is incomplete without the other half of the sentence: with SSD-streamed MoE experts, enough free disk, and published testing centered on specific Apple Silicon machines.

The next useful milestone would be a reproducible benchmark set across several M-series Macs using the same prompts, context lengths, and storage configurations. Right now, the project’s best published grounding remains its Hugging Face deployment page and GitHub repository.

Key Takeaways

  • Gemma 4 26B-A4B is not best supported as a true 2 GB all-in-memory Mac model; the strongest citable project figure is about 3 GB resident RAM for Gemma-only with SSD-streamed experts.
  • The low-memory result depends on SSD offload, so the full model is not loaded into RAM at once.
  • Standard local Apple Silicon setups are materially heavier, at about 6 GB with llama.cpp memory mapping and about 13 GB for vanilla 4-bit MLX in the project’s own comparison.
  • The model files themselves remain large, with cited quantized files ranging from about 9.97 GB to 16.6 GB and a 14.2 GB IQ4_XS example.
  • The project’s own published guidance recommends 16 GB+ RAM and about 30 GB of free disk, which is narrower than “any M-series Mac.”

Further Reading

Frequently Asked Questions

Can Gemma 4 26B really run in 2 GB RAM on a Mac?

Not on the best citable evidence here as a full in-memory setup. The closest published figures from the project behind the claim put Gemma-only resident memory at about 3 GB, and that result depends on SSD-streamed experts rather than loading the whole model into RAM.

What stays in memory and what stays on disk?

The project’s GitHub repository says hot experts and a small cache stay resident in DRAM, while cold experts are streamed from flash storage on demand. That is why resident RAM can stay low even when the actual model files are far larger.

How much RAM does Gemma 4 26B usually need on Apple Silicon?

In this source set, the range runs from about 3 GB resident RAM with Expert Sniper, to about 6 GB with llama.cpp memory mapping, to about 13 GB for vanilla 4-bit MLX on the project’s own comparison page, with one downstream MLX report at about 26 GB. The runtime matters as much as the model.

Is the 2 GB claim useful for real Mac buyers?

Only partly. A resident-memory number is useful if you are debugging whether a runtime will fit, but it is not a full workflow spec. The same deployment page also recommends 16 GB+ RAM and about 30 GB of free disk, which is the more practical guidance.

References

Last reviewed: 2026-08