A capable 30B model now fits in 17GB, and Apple just shipped a $899 box that can hold it. Read the second number before you buy.

For two years the honest answer to “can we run AI on our own hardware instead of paying per token” has been: yes, and you will not enjoy it. The models small enough to run were not good enough to use, and the models good enough to use needed hardware priced like a car. We said as much in our post on when local deployment is the right answer and again in the total cost breakdown.

Two announcements this month moved that line, and they moved it from opposite directions.

On August 10, Meta released Muse Glimmer, a 30-billion-parameter dense model under the Apache 2.0 licence, built specifically for local agent work: tool calling, coding, long multi-step sessions. At 4-bit quantisation the weights come down to roughly 17GB, and Meta’s own guidance is that you want 24GB to 32GB of memory or VRAM to run it properly.

On August 25, Apple announced new Mac minis and Mac Studios with the M6, M5 Pro, M5 Max and M5 Ultra, and pitched them at local AI directly, including clustering multiple machines over Thunderbolt 5 “to run large AI models entirely on device.”

So the model got small enough and the hardware got cheap enough at the same time. That is real. What follows is the part the announcements do not help you with, which is deciding what to buy.

The two numbers that matter, and only one of them is on the box

Running a model locally has two separate bottlenecks, and confusing them is how people end up disappointed by a machine that benchmarked well.

Memory capacity decides whether the model runs at all. The weights have to fit, plus the KV cache for your context, plus the operating system. Too little and it either refuses to load or spills to disk and becomes unusable. This is a hard yes-or-no gate.

Memory bandwidth decides how fast it generates. Producing each token requires reading the model’s weights out of memory. That makes generation memory-bound, not compute-bound, on essentially all consumer hardware. The ceiling is simple arithmetic:

tokens per second ≤ memory bandwidth ÷ size of the weights in memory

Real throughput lands somewhere around 50 to 80 percent of that ceiling once you account for the KV cache, sampling overhead and the fact that nothing runs at theoretical peak. But the ceiling is a hard ceiling. No amount of GPU cores moves it.

Now look at what Apple actually claimed. The M6 Mac mini delivers “up to 13.5x faster LLM prompt processing in LM Studio” versus an M1 mini, and 4.8x versus M4. The M5 Max Mac Studio: “up to 10.7x faster LLM prompt processing” versus M1 Max.

Prompt processing is the other half of the job: chewing through the input before the first token comes out. It is compute-bound and it parallelises well, so it is genuinely where the Neural Accelerators help, and Apple’s numbers are probably fair. It is also not the number you feel when you are waiting for an answer to finish printing. Apple is quoting the half of the workload its new silicon improved most. That is normal marketing and it is not dishonest, but it means the headline figure does not tell you how fast the thing will actually talk to you.

The arithmetic, for Glimmer at 4-bit

Taking 17GB of weights and each machine’s published bandwidth, here is the generation ceiling. These are upper bounds, not measurements, and you should expect real figures meaningfully below them.

Several things fall out of that table that no product page will tell you.

The $899 Mac mini is not the $899 Mac mini. It ships with 16GB, and 16GB does not hold a 17GB model. You need the 32GB configuration, which is a paid upgrade, and Apple’s memory upgrades have never been cheap. That is worse this year than usual for the reasons we covered in the RAM shortage post: memory pricing is the tightest it has been in a decade, which is also why NVIDIA raised the DGX Spark from $3,999 to $4,699 in February. Budget for the upgrade, not the headline.

A maxed M6 mini works, but it is the floor. Around 10 tokens per second as a ceiling means realistically 6 to 8. That is fine for a background agent chewing through documents, summarising overnight, or answering the occasional question. It is slower than reading speed, so it will feel sluggish for anything interactive.

The M5 Pro mini at $1,699 is the sensible entry point. Nearly double the bandwidth of the M6 and 64GB available, which leaves headroom for longer context and for the model to grow. If you want one machine to try this on and you are not sure it will work out, this is the one to buy.

The DGX Spark’s problem is now clearer. We wrote about it in the two Sparks post and this is the sharpest illustration yet: 128GB of capacity at 273 GB/s. It will hold models a Mac mini cannot dream of, and it will generate from them slowly. It is a machine for developing against big models, which is what NVIDIA says it is for. It is not a machine for serving them fast, and at $4,699 an M5 Max Mac Studio at $2,499 will out-generate it on anything that fits in both.

The RTX 5090 wins on speed and loses on ceiling. 1,792 GB/s is more bandwidth than anything else on this list, and 32GB of VRAM caps what you can put behind it. Glimmer at 4-bit fits with room to spare. The next model up will not. The power and heat realities we measured still apply, and a 575W card in an office is a different proposition from a fanless mini.

The M5 Ultra is the only consumer machine that gets both. 512GB of unified memory at 1.2 TB/s has no equivalent at any price outside a datacentre. It is $5,499 to start, the 512GB configuration does not arrive until late October, and it is genuinely the machine you buy if you intend to run large open models in-house as infrastructure rather than as an experiment.

Is Glimmer actually good?

The honest answer is that we do not know yet, and neither does anyone quoting the launch post.

Meta claims Glimmer outperforms Gemma4-31B and Qwen3.6-27B on agentic, coding, multimodal, safety and reasoning benchmarks, and cites SWE-Bench, DeepSearch QA, MCP-Atlas and tau-Bench. Those are the right benchmarks for what the model claims to be. They are also Meta’s own numbers on a model Meta released eighteen days ago, and vendor benchmark tables have a consistent historical bias in one direction.

What is verifiable and matters more:

• The licence is Apache 2.0. Not a community licence with a user-count threshold and a naming clause. You can use it commercially, modify it, and ship it inside a product without a lawyer reading anything. For a small business this is a bigger deal than two points on a benchmark.
• The runtime support is already there. llama.cpp, MLX, ExecuTorch, Ollama, LM Studio, vLLM and SGLang. You are not compiling anything unusual, and MLX in particular means it is a first-class citizen on the Apple hardware above.
• It is dense, not mixture-of-experts. All 30 billion parameters are active for every token. That is worse for speed than an MoE of the same nominal size, and it is why the bandwidth arithmetic above bites as hard as it does. It also makes the memory requirement predictable, which is why it fits the “runs on one machine” pitch cleanly.
• The quantisation claim is the load-bearing one. Meta says 4-bit costs “minimal to no degradation on agentic tasks.” Treat that as the thing to test first, because everything above depends on 17GB rather than 55GB, and quantisation damage tends to show up in exactly the long multi-step tool-calling sessions this model is sold for.

Test it on your own work before you buy hardware for it. Rent an hour of a big GPU, or run the quantised version on whatever machine you already have and accept that it is slow. The question is never whether a model is good in general. It is whether it is good enough at your specific task to be worth not paying an API.

When this is worth doing, and when it is not

Nothing here changes the underlying decision, which we laid out before and still stand behind. Local inference is worth it when one of these is true:

• Data cannot leave. Clinical records under PHIPA, legal files, anything under a contract that names where data may be processed. This is the reason that actually justifies the work, and it is not really about cost.
• Volume is high and predictable. A steady background workload, running most hours, on a task where the model does not need to be the best available. This is where the arithmetic beats an API, and it needs real volume to do so.
• You need it to work offline, or with latency you control.

It is not worth it when your usage is spiky, when you need frontier-model quality, or when nobody on the team wants to own a machine. A box that runs a model is a box someone has to patch, monitor and eventually replace, and unattended internal infrastructure is exactly how organisations end up in the situation we described in the post on self-hosted tools landing on CISA’s exploited list. Add that ownership to the cost, honestly, before you compare it to a monthly API bill.

What we would do

If you are curious and have not tried it: install LM Studio or Ollama on the best machine already in the building, pull Glimmer at 4-bit, and point it at a real task you would otherwise pay for. You will learn more in an afternoon than from any table, including ours.

If that goes well and you want dedicated hardware: an M5 Pro Mac mini with 64GB. It is $1,699, it is quiet, it draws almost nothing, and it is fast enough to be pleasant rather than merely possible.

If you already know you are doing this at scale, and data residency is the reason: wait for the 512GB M5 Ultra in late October and size it for the models you expect in a year, not the one you are running now. Memory is the constraint that ages worst and the one you cannot add later on Apple silicon.

And if you are not sure which of those three you are, talk to us. We have made this decision for our own workloads and for clients, and the most common right answer is still “not yet, keep using the API, revisit in six months.” That is a fine outcome and it costs nothing to establish.

Sources

• Introducing Muse Glimmer: An Open Agentic Model That Runs on Your Device, Meta AI Research, August 10, 2026, for parameters, licence, memory requirements, runtimes and benchmark claims
• Meta Open-Sources Muse Glimmer: a 30B Local Agentic Model Optimised for On-Device Execution, InfoQ, for the dense architecture, quantised memory figures and recommended hardware
• Apple unveils a more powerful Mac mini featuring the all-new M6 and M5 Pro, Apple Newsroom, August 25, 2026, for specifications, prices, the prompt-processing claims and Thunderbolt 5 clustering
• Apple introduces Mac Studio with M5 Max and M5 Ultra, Apple Newsroom, August 25, 2026
• New Mac mini M6 and M5 Pro: Everything you need to know, Macworld, for the per-configuration memory bandwidth figures
• NVIDIA DGX Spark, NVIDIA, for GB10 specifications and unified memory bandwidth
• NVIDIA GeForce RTX 5090 specifications, NVIDIA, for 32GB GDDR7 and 1,792 GB/s