LLM Sizer — Methodology & Assumptions

Version 0.17.3 · 2026-10-19 · Status: public, early build, open source. This is the public, plain-language explanation of every number LLM Sizer shows. It is the single source of truth: the tool’s About page renders it and the open-data repo’s FORMULAS.md is a nightly copy of it, and any change to a formula, constant, or data source in the tool must be reflected here first (that rule is part of the project’s definition of done). Comments and corrections: the feedback form in the tool, or an issue on the open-data repository. Changelog at the bottom.

1. What the tool answers — and what it doesn’t

LLM Sizer answers three questions for a machine you own or are considering: which open AI models fit in its memory, how fast they will write answers, and what you’d have to change to make a model fit. Every answer is a calculation from published data, calibrated against real measurements, and every estimate is labeled as one.

It does not run a benchmark on your machine, judge model quality (we show quantization-quality notes where quantizers publish them, nothing more), or cover image/video generation models (those are compute-bound and need a different method).

How to read the table

Machines are rows (one row per memory size, grouped by model line: “Mac mini · M6” covers the 16, 24 and 32 GB configurations), models are columns (one column per model at a chosen quantization and context length). Each cell is one of three marks:

• Gold sphere: runs. The whole configuration fits in the memory available for AI with at least 10 % headroom. The sheet behind the cell says how much of the available memory it uses.
• Gold ring: runs with a compromise. Something has to give, and the cell already picked the cheapest concession by our ranking (section 5): close the apps you reserved memory for, raise the macOS memory limit, shorten the context, or drop to a smaller build, in that order of preference. A cell that fits with under 10 % headroom is also a ring, labelled tight. The sheet lists the chosen fix, up to three alternatives (never “an earlier option plus something extra”), and applies any of them to the column with one tap.
• Grey bar: doesn’t fit. Nothing at or above the quality floor (2-bit builds by label, adjustable in Advanced) fits even with every concession. The sheet shows the nearest miss: the smallest allowed build and how much memory it would still need.

Everything that configures the chart lives in the Settings panel (a column on the right on wide screens, a button above the chart on narrow ones): the machines and which of their memory sizes to show, the models (each column is one model at one build and one context, set when you add it and editable in the list, which you can also reorder by dragging), and two switches. “I’ll also use it for work” reserves 16 GB for your own apps (adjustable under Advanced); it is off by default, so the default table is the best case with nothing else running. “macOS memory limit” shows the share of RAM macOS lets the GPU use (macOS 26’s share for that size, from 67 % on 8 GB to 91 % on 512 GB); the sheet’s fix and Advanced can lift it to the override (section 4). The context cache is counted at FP16 by default, as every runtime stores it unless told otherwise (section 5). Speed numbers are always for the configuration that fits: a ring’s speed is the speed of its fix, not of the build you asked for. Column headers carry the model’s total and active parameters (“180B · 6.7B active” for a mixture-of-experts model), the build (Q4 by default) and the context length (32K by default). Seven tiles above the chart switch between the fit table, the speed bars for one machine, the every-chip grid (“How fast is every Mac?”), which puts one model at one build and context on every Apple chip in the catalog, generations across and base, Pro, Max and Ultra down, as writing speed or as the wait before the first word for a prompt of that length (section 6.4), coloured by the feels-like bands; each chip shows either the biggest memory it was sold with (its fastest version) or the smallest that runs the model as asked, an outlined chip runs only with the memory limit lifted and a dashed one has no measured runs of its own; tapping a chip lists every memory size it came in with its chip version, both speeds and the list price, because memory size changes a chip’s speed only through the version that comes with it: writing follows memory bandwidth (the 36 GB M5 Max is the 32-core chip at 460 GB/s), reading follows GPU cores (the 96 and 256 GB M5 Ultra read on the 64-core GPU their list price buys), and beyond that more memory changes what fits, not the speed; a chip the catalog adds later takes its place in the grid by its name, the buying view (“Which machine?”), which places every current machine configuration on a map of memory size (rows) against estimated writing speed (x, shaded into the feels-like bands of section 6.5) for one model at one build and context, with a Q4 / Q8 switch on a shared axis, a minimum speed (any, 5, 12, 30 or 60 tok/s), and, from Settings, a budget and a laptop-or-desktop preference; a configuration qualifies when it fits as asked, has a speed estimate at or above the minimum, costs no more than the budget and matches the preference, and the map names the cheapest qualifying configuration with a current list price on its mark; older catalog machines you put in the table appear too, without a price; a compromise is a ring at its fix’s speed and never qualifies, a machine that does not fit is a dash at the left edge and is never placed at a speed, a fit with no speed estimate never qualifies, an unpriced machine (a custom one, or a size not on sale yet) is listed but never called cheapest, and when nothing qualifies a line under the map says so and names the nearest misses instead of inventing a pick; every mark opens the sheet, which carries the machine’s list price, the memory bar and the speed with its estimate detail on demand, the memory chart, which shows what each model needs at its build and context, split into weights, context cache, buffers and, for MLX files at long context, prompt scratch, with one machine’s limit drawn as a reference line, and the machine view (“What can it run?”), which takes one machine at one memory size and lists every model in the catalog, grouped by the size of its Q4 file (under 40 GB, 40 to 120, 120 to 256, 256 to 512, over 512), with its parameter count, the highest-quality build that runs as asked at 32K, the memory that configuration needs on the machine, and the longest context that runs at Q4; when nothing runs as asked it names the cheapest compromise or the nearest miss. “Show every build and context” turns each model into a table whose rows are the quantization buckets and whose columns are the context windows, with the same three marks and, under each, the speed and the memory needed; a build the model has no file for is shown as a gap, never as a substitute file. The seventh tile, “Every way to 128 GB” (section 6.6), puts one model at one build on every route to about 128 GB of memory, from a gaming PC with the model’s experts in system memory to Macs, shared-memory boxes, linked Sparks and workstation cards, with the price, the memory speed, the memory the AI can use, the fit, the writing speed, a 1,000-word answer’s time and, where a run measured it, the time to read 50 pages. A machine group can be linked: picking a machine that is already in the table adds a second identical one pooled with it (up to four; also the ×1 to ×4 chips in Settings), and the row then reads “2 × NVIDIA DGX Spark · 128 GB each · 256 GB pooled” with the summed price, on every view; a layer split | tensor parallel switch says how the model is split across the pool (section 4 for the memory, 6.1 and 6.4 for the speeds).

Every table state is a link (?s=), the share image is rendered from the same math on the server, and a custom model or machine you type in travels inside the link so the person you send it to sees exactly your table.

Every number in the sheet behind a cell links to its source: the weights to the file’s repository on Hugging Face, the machine to its row in the sources table (bandwidth, prices, spec pages), the context cache and the available memory to the sections of this page that compute them, and the speed to the chip profile it used (effective bandwidth and fixed cost per token), with a sentence saying whether that profile was fitted from this chip’s own rows, assumed, or taken from the chip’s tier.

What kind of number is this

Every figure the tool shows is one of five kinds, and the sheet behind a cell says which:

• Exact file data. Weight sizes are the bytes Hugging Face reports for the file you would download; machine memory and bandwidth are the makers’ specifications. Nothing is estimated.
• Derived from the architecture. The context cache is computed from the model’s config.json (attention design, layers, heads) and the context length: exact for a design we can read, the pessimistic classic formula for one we cannot (“architecture assumed conventional”).
• Fitted to measurements. Speed. Each chip’s effective bandwidth and fixed cost per token are fitted to the community llama.cpp table; the architecture and runtime terms are fitted to measured runs and judged on runs the fit never saw (section 6.1). Every fitted term carries the number of rows it rests on, and a term with too few rows is labelled assumed.
• Quoted. A special build’s published speed. It is shown as a note with its source, never as the cell’s number: one measurement on one chip cannot be carried to the others, and the tool has no per-token read to reason from for a build that streams from the SSD.
• Policy margins. The reserves for macOS and your apps, the working buffers and the MLX prompt scratch are safety margins chosen from owners’ observations, not measurements of your machine (sections 4 and 5). They are the numbers most worth arguing with, and section 8 says how far off they have been seen to be.

2. Words we use

• Model / weights — the file you download. Its size is the single biggest number in every fit calculation.
• Quantization (Q4, Q8, FP8, MLX 4-bit…) — storing the weights with fewer bits per number. Q4 ≈ 4.5 bits per weight and is the common local default; Q8 ≈ 8 bits and is near-lossless; FP8/BF16 are the “full” formats. Smaller quant = smaller file = faster, with some quality loss below Q4.
• Context window — how much text the model can hold “in mind” at once (your prompt + its answer + any documents). Measured in tokens; 1 token ≈ ¾ of an English word.
• Context cache (KV cache) — memory the model uses to remember the context while it works. It grows with the context length and is the part most fit charts ignore.
• Unified memory — on Apple silicon (and boxes like the DGX Spark), CPU and GPU share one pool. A model must fit in the part the GPU is allowed to use.
• Bandwidth (GB/s) — how fast the chip can read its own memory. It decides writing speed.
• Tokens per second (tok/s) — how fast the answer appears. Decode = writing the answer; prefill = reading your prompt before the first word appears.
• Active parameters — in mixture-of-experts (MoE) models only part of the model works on each token; that part is what gets re-read per word and what sets the speed.

3. Where the data comes from, and how fresh it is

How models get in. Every night a job lists what the five quantizer organisations above published or updated, finds the maker’s original repository for each (from the quantizer’s own “base model” note, otherwise by name), and reads that model’s config.json and file list. It keeps everything from the last twelve months that has at least a billion parameters and a thousand downloads; the ~25 featured models (hand-picked, current releases only) are always refreshed. Repositories that are edited variants — uncensored or “abliterated” builds, mixed-precision experiments, draft models — are skipped by name and logged. A model released today is normally in the tool tomorrow, with its real file sizes.

What can go wrong, and the safety net. Before new data goes live it is compared with what is live now: if a featured model lost its quantizations, a known file changed size by more than a tenth, a model’s architecture could suddenly not be read, or the catalog shrank, the update is held and a person looks at it. The page always shows the date of the data it is using. The full export (search index, one file per model, machines, benchmarks, factors, this document) is mirrored nightly to the public data repository.

4. How much memory is actually available for AI

Units first. Memory sizes are quoted in binary gigabytes: a “32 GB” Mac has 32 GiB, which is 34.4 GB in the decimal gigabytes file sizes use (× 1.0737). The tool converts the machine’s size before subtracting anything, so a 48 GB MacBook Pro is 51.5 GB, a 128 GB machine 137.4 GB, a 512 GB Studio 549.8 GB. The DGX Spark’s 64 and 128 GB are binary too (68.7 and 137.4 GB).

Macs (Apple silicon). macOS does not let the GPU use all of the unified memory. On macOS 26 the default share grows with the machine: 67 % on 8 GB, 74 % on 16 to 24 GB, 78 % on 32 to 48 GB, 81 % on 64 and 96 GB, 84 % on 128 and 192 GB, 87 % on 256 GB, and all but 48 GB on 512 GB (464 of 512, 91 %). These are Metal’s recommended working set (recommendedMaxWorkingSetSize) as logged on macOS 26 in public llama.cpp, Ollama, MLX, oMLX and LM Studio reports, more than 50 of them across every size, plus a reading on the maintainers’ 128 GB M5 Max on macOS 26.6.2 (115,448,725,504 bytes). Where a log prints the full byte count, it is exactly a whole percentage of the memory rounded up to a 16 KB page, so Apple uses a table by size rather than a formula. The values are the same from the macOS 26 betas of August 2025 through 26.6.2, and on macOS 27; Apple does not document them. The share steps up once per doubling, and the tool applies each step from 16, 32, 64, 128 and 256 GB, which fits every size Apple sells; for a custom size between two of them the step is an assumption. Macs still on macOS 15 get less: two thirds under 36 GB, three quarters from 36 GB, and in recent versions never more than 48 GB held back (208 GB on 256, 464 GB on 512). The tool assumes macOS 26; on macOS 15, set 67 % or 75 % with the memory-limit slider in Advanced. Power users can raise this limit from Terminal (sudo sysctl iogpu.wired_limit_mb=…) — it works and it’s common, but it’s unsupported by Apple, so the tool treats it as an explicit toggle, off by default.

On top of that, the machine has to run macOS itself and — if it’s your everyday computer — your apps. We assume:

• macOS base: 6 GB with the default GPU limit. Once the limit is lifted, macOS is the only reserve left and on big machines it keeps more: the larger of 6 GB and 5.5 % of the memory (a 512 GB Studio keeps about 30 GB, which is what owners running with the override report). A policy margin.
• Your apps (the “I’ll also use this machine for work” toggle): a budget of 8 GB (light) / 16 GB (typical, the default when the toggle is on) / 24 GB (heavy) — a browser with many tabs alone is 5–7 GB. Adjustable in Advanced.

available = min( RAM × GPU-limit ,  RAM − 6 GB − apps )          default limit; RAM in decimal GB (nominal × 1.0737)
available = RAM − max( 6 GB , 5.5 % × RAM ) − apps                with the override

The min matters: on machines of 128 GB and up the share macOS holds back already covers the OS and your apps; on smaller machines it doesn’t, and your apps become the real limit. Example: a 32 GB Mac mini (34.4 GB) used for work → min(34.4 × 78 % = 26.8, 34.4 − 6 − 16 = 12.4) → 12.4 GB for AI; with nothing else running, 26.8 GB. That’s why a model that “fits 32 GB” on a spec sheet doesn’t fit a 32 GB machine you work on. Available memory by size, nothing else running: 8 GB → 2.6 · 16 → 11.2 · 24 → 19.1 · 32 → 26.8 · 36 → 30.2 · 48 → 40.2 · 64 → 55.7 · 96 → 83.5 · 128 → 115.4 · 192 → 173.2 · 256 → 239.1 · 512 → 498.2; at 8 and 16 GB the 6 GB for macOS runs out before the GPU limit does. With the override: 32 → 28.4 · 64 → 62.7 · 128 → 129.9 · 256 → 259.8 · 512 → 519.5.

Other platforms — see §7.

Linked machines (0.13). Two to four identical machines can be pooled: Macs over Thunderbolt 5 (llama.cpp’s RPC mode, exo, MLX distributed), DGX Sparks over their 200 Gb/s ConnectX ports (two directly, three in a ring, four through a 200 Gb/s switch; NVIDIA’s Cluster Assistant), or cards in one box. The pool’s available memory is the sum of what each machine gives on its own: each keeps its own macOS or Spark reserve and its own GPU limit, judged on its own size (two 24 GB minis get 74 % each, not the 78 % of a 48 GB machine), the override lifts each machine’s limit, the work-apps reserve is taken once (you sit at one machine), and the link takes 2 GB per machine for its buffers, a policy margin.

available (pool) = min( N × RAM × GPU-limit ,  N × (RAM − 6 GB) − apps ) − 2 GB × N     Macs, default limit
available (pool) = N × (RAM − 8 GB) − apps − 2 GB × N                                  Sparks and other unified boxes

Two DGX Sparks: 2 × (137.4 − 8) − 4 = 254.9 GB. Two 128 GB M5 Max Studios: min(2 × 137.4 × 84 %, 2 × (137.4 − 6)) − 4 = 226.9 GB at the default limit, 255.8 GB with the override on both. The community raises the wired limit on every machine of a Mac cluster (speeds collapse otherwise), so a pool that only fits with the override is shown as a ring with that fix, as a single machine would be. Runtimes report a usable share of the Spark’s 128 GB nearer 105 GB under vLLM (page cache and the 0.8 to 0.9 memory utilisation setting); the tool keeps the 8 GB reserve of section 7 and the 2 GB link margin, and section 8 records the gap.

5. How much memory a model needs

needed = weights + context cache + working buffers + prompt scratch (MLX files above 8K of context)

• Weights = the published file size of the quantization you picked (not an estimate).
• Working buffers = 1.5 GB + 1 % of weights — activations and the runtime’s scratch space. A policy margin set from owners’ peak readings: about 1.1 GB above the file on a 17 GB dense model, about 0.4 GB above weights and cache on a 224 GB mixture-of-experts file (the earlier 5 % of weights rule charged that machine 11 GB).
• Prompt scratch (MLX files only) = 0.15 GB per 1K tokens of context beyond 8K. MLX allocates working memory to process a long prompt in one pass, and owners’ peak readings grow at about that rate on a 27B and a 397B model between 8K and 128K (3.6 GB at 32K, 18 GB at 128K); GGUF runtimes process the prompt in batches inside the buffers. A policy margin; it is its own segment in the memory bar and follows the file that runs, so a GGUF fix on an MLX column drops it.
• Context cache = bytes per token × context length. Bytes per token depends on the model’s attention design, which we read from its config:

bytes is the cache precision: FP16 by default (2 bytes), which is what llama.cpp, LM Studio, Ollama and MLX store unless told otherwise; 8-bit (1 byte) or 4-bit (0.5) selectable in Advanced. The sheet says so under the memory bar, with what an 8-bit or 4-bit cache would make of the number, because owners who run a quantized cache see half or a quarter of the tool’s context cost and asked. Most 2026 models use compressed or hybrid attention, so long context is far cheaper than older fit charts assumed. Models whose design we can’t read from the config are marked “architecture assumed conventional” and use the classic formula — the pessimistic choice.

Across linked machines (0.13). The shards pack looser than the file: 5 % more weights on a layer split (exo split a 540 GB model into 2 × 280 GB) and 8 % under tensor parallel (Kimi K3 in four-way tensor parallel on M3 Ultras held 420.8 GB per machine against a 363 GB share). Under tensor parallel the context cache is duplicated on every machine for latent-attention models (the one latent head cannot be split: measured on Kimi K3 and GLM-5.2), and split between the machines, so unchanged in total, for grouped-query models (assumed). Each machine keeps its own working buffers (1.5 GB each, plus the 1 % of the weights once) and, on MLX, its own prompt scratch. The whole download has to sit on every machine’s disk under tensor parallel; the sheet says so, the memory bar does not count it.

Verdicts:

• Runs (solid dot) — needed ≤ 90 % of available.
• Tight (ring) — it fits, but with under 10 % headroom: 90 % < needed ≤ 100 %. Real sessions grow; treat it as “will work, watch it.”
• Runs with a compromise (ring) — the asked-for configuration doesn’t fit, but a change does, and the cell says which. The tool tries, alone and in combination: closing your work apps (only when the work toggle is on), the macOS memory-limit override (only on Macs, and only when the GPU limit is what binds), a shorter context (the next chips down: 128K → 32K → 8K), a smaller quantization of the same format, and any special build that streams from the SSD (below). Every combination that fits gets a cost — closing apps 1 · the override 2 · each context step 4 · a smaller quant 8 (9 under 3 bits) · a special build 16 · tightness ½ — and the cheapest wins; ties go to the higher-quality (more bits) and longer-context option. Up to three genuinely different alternatives are kept for the tooltip (“or: at 32K context instead of 128K”).
• Doesn’t fit (dash) — nothing allowed fits at any context. The cell still explains itself: “the smallest allowed build, UD-IQ2_XXS (2-bit) at 711 GB, needs about 724 GB at 128K; this machine offers 520 GB at most.”

The quality floor. The automatic search never proposes a build under 2 bits per weight, judged by the file’s label (IQ1_* / Q1_* are out; IQ2_*, Q2_*, MLX-2bit are in). The label is used rather than the measured bits-per-weight because dynamic quantizations keep some tensors at higher precision — a 1-bit-labelled build of a large mixture-of-experts model reads as 2.3 effective bits. When only a sub-floor build would load, the dash says so (“a 1.3-bit build (466 GB) would load, quality unknown”); Advanced lets you lower the floor and pick such builds by hand.

Special builds. A few community builds are not plain quant files. Qwen 3.8 Flash Next is a 125B mixture of experts plus a 51B-parameter n-gram table (Qwen calls it the engram memory): the model hashes the last few tokens and reads about 2.7 KB from the table once per token, at one address, so the table can stay on the SSD and be paged in as needed while the experts stay in memory. The AtomicChat builds do exactly that, and the tool carries all three as hand-maintained entries with two sizes each: what stays in memory and what streams from the SSD. IQ4_XS · engram on SSD keeps 45.8 GB in memory and 39.1 GB on the SSD (84.9 GB to download), Q4_K_M · engram on SSD 54.5 + 38.4 (the publisher’s recommended file), Q5_K_M · engram on SSD 56.1 + 54.4. The memory need is the resident part plus the usual cache and buffers; the SSD part is shown beside the memory bar and never counted in it, and a cell that runs such a build says so in a flag (it also needs that much free SSD space, a llama.cpp with qwen4exp support, mmap on, -fit off and --jinja, per the publisher). You can pick one directly, in the column’s build list under the files (“Q4 · engram on SSD · 45.8 GB in memory”), as the “SSD build” segment of the buying map, and the machine view names it as the best build when no file fits as asked; the fix search still offers the smallest one as a compromise for a Q4 ask, at cost 16: “with the engram-on-SSD build (45.8 GB in memory, 39.1 GB streamed from the SSD)”. Their speed is not estimated: the read per token is the active experts plus SSD lookups the model has no rows for, and the one published measurement (36 tok/s writing, 518 tok/s reading, on an M5 Max MacBook Pro with 64 GB and the memory limit raised to 56 GB) cannot be carried to other chips. The sheet quotes it as a note with its source; the speed view says “speed not estimated”; on the buying map such a machine sits just right of the zero line, qualifies when any speed will do, and never meets a minimum.

The n-gram table in an MLX file. Mac engines do the same with every MLX file of Flash Next: oMLX, MTPLX and mlx-serve leave the table on the SSD and read the few rows each word needs from there. The tool counts an MLX file of this architecture that way on a Mac. The table’s share of the file (51 of 180B parameters, 28.3 %) is shown beside the memory bar as the SSD part, and the rest is the weights. The 111.5 GB mlx-community 4-bit file keeps 80.0 GB in memory and 31.6 GB on the SSD, and needs 82.5 GB at 8K, 86.7 GB at 32K and 103.5 GB at 128K, so it runs on a 128 GB Mac at the default limit (115.4 GB) in the fit table, the buying map and “What can it run?”, with a flag that says where the table is. oMLX’s measured peaks sit under those numbers: 75.5 GiB (81.1 GB) on the same file at 4K, and 69.5 to 74.6 GiB (74.6 to 80.1 GB) from 4K to 128K on oMLX’s own 106.3 GB oQ4e build (section 8, item 14). The rule is keyed on the architecture, the config’s qwen4_exp model type that also gives Flash Next its speed class, so it reaches the live model records without waiting for the nightly export. It does not apply to GGUF files, which keep the table inside the weight shards (the standard 119 GB Q4_K_M needs all of it and does not fit a 128 GB Mac at the default limit; only the AtomicChat builds above split it out), to an engine that loads the table into memory (oMLX can; the oQ4e build then peaks at 99 to 105 GiB), or to an MLX file counted on a machine with no MLX engine, such as the DGX Spark, where the whole file counts as before. The 128 GB comparison (section 6.6) uses the same rule.

Such a file gets the normal speed estimate, unlike the GGUF builds above. The table costs a few kilobytes per word against about 3.7 GB of active weights, and the oMLX runs the MLX terms are fitted and checked on ran with the table on the SSD (11 of the 13 M5 Max submissions behind its 68.9 tok/s row). The GGUF builds keep “not estimated”: their one run is on a 64 GB machine at a raised limit, and no llama.cpp row with the table on the SSD is in the fit.

6. Speed

6.1 Writing speed (decode) — the number that matters day to day

To write each token, the machine re-reads the model’s active weights plus the context cache, and then does a fixed amount of work that does not shrink with a smaller file: launching the kernels of every layer, routing between experts, attending over the context. The time per token is the sum of the two, and the speed is its inverse:

GB per token = active_params × bytes per weight (quant) + cache bytes per token × attended context
ms per token = read + fixed + context
    read     = GB per token ÷ effective bandwidth × 1000 × read factor   (K-quant GGUF files on Apple silicon 1.25, everything else 1)
    fixed    = (chip fixed cost + architecture cost) × MLX factor         (GGUF files 1; section 6.2)
             = expert layers × cost per expert layer of the chip generation  (MLX, mixtures under 6B active; section 6.2)
    context  = (attention cost + MLX attention cost) × attended context ÷ 32K
               + MLX index cost × context ÷ 32K                           (sparse-attention models on MLX only)
tok/s        = 1000 ÷ ms per token
attended context = the context, or on MLX a sparse-attention model's budget when the context is longer (2,048 tokens for Qwen 3.8 Flash Next)

Linked machines (0.13). Whether a second machine adds speed depends on how the model is split, not on the hardware, so the tool asks:

layer split      ms per token (pool) = ms per token (one machine, the whole model) + hop × (N − 1)
tensor parallel  ms per token (pool) = ms per token (one machine, the whole model) ÷ N + C × log2(N)

On a layer split (llama.cpp’s RPC mode on any link, MLX pipeline, exo pipeline) every token passes through every machine in turn, so the pool does one machine’s work for the whole model plus a hop per extra machine: the published series fit 4.0 ms per hop on Macs (exo on M4 Pro minis, 49.3 → 44.4 → 39.7 tok/s at one, two and three; llama.cpp on four M3 Ultra Studios over Thunderbolt, 20.4 → 17.2 → 15.2; four points, range 2.2 to 9.1) and 3.9 ms on Sparks (llama.cpp RPC on two Sparks: gpt-oss-120b 57 → 47, Qwen3-235B 15.51 → 14.57). GPU boxes take 1.0 ms, assumed. A model that fits one machine therefore writes slower across two. Under tensor parallel (MLX over Thunderbolt 5 RDMA on macOS 26.2+, vLLM or TensorRT-LLM with NCCL over the Spark’s ports, PCIe in a GPU box) every machine reads its shard at once, and the synchronisation cost grows with log2 of the count: 4.5 ms per doubling on dense models (three M5 Max laptops on Qwen3.6-27B, 18.68 → 32.59 → 40.79 tok/s; one, two and four Sparks on the same model under vLLM, 12.63 → 22.57 → 33.11; four points, range 3.9 to 5.2) and 10.9 ms on mixtures of experts (exo on four M3 Ultras: Qwen3-235B 19.5 → 26.2 → 31.9, DeepSeek V3.1 21.1 → 27.8 → 32.5; TensorRT-LLM on two Sparks; range 9.3 to 12.5), because the expert gather does not shrink with more machines. GPU boxes take 1.5 ms per doubling, assumed from one RTX PRO 6000 pair. Tensor splits need the count to divide the model’s heads and hidden sizes, so the tool offers them at 2 or 4 machines, and only where the link is fast (Thunderbolt 5 on the M3 Ultra, the M4 Pro and Max and the M5 family; every Spark; every GPU box). calibrate.py fits both constants from benchmarks.json › cluster_series into factors.json › cluster, with the count of series and the range, and the sheet says which applied and whether it is fitted or assumed. Nobody has measured an M5 Ultra or an M6 cluster; the Studios ship on 2026-09-22.

Until version 0.8 the tool used bandwidth × efficiency ÷ GB per token, one efficiency factor per chip. That factor was calibrated on a 3.8 GB read (Llama-2-7B at Q4_0) and carried the fixed cost inside it, so it was wrong in both directions away from that size: models with a small active set (a 2 to 4 GB read per token) came out 1.5 to 3 times too fast, and 8-bit dense models on the Ultra chips 0.55 to 0.85 times too slow. The fixed-cost model separates the two.

Chip profiles are fitted, not assumed. For every chip, calibrate.py takes the F16, Q8_0 and Q4_0 rows of the community llama.cpp table (13.5, 7.2 and 3.8 GB reads of the same 7B model) and fits a straight line of milliseconds per token against gigabytes read: the slope is the effective bandwidth, the intercept the chip’s fixed cost. The effective bandwidth is stored as a share of the chip’s specification, because that share carries across the GPU bins of one chip (an M4 Max 32-core at 410 GB/s fits 0.88, the 40-core at 546 GB/s 0.90; an M3 Max 30-core 0.96 against the 40-core’s 0.98) while the GB/s does not.

Two-row chips (M1, M3, M5, the Spark) have a line through two points and no slack; the rest fit within 2 % except the M1 Max (8 %, six rows from several machines) and the M5 Max (6 %). The Ultra chips read at 80 to 88 % of their specification: the multi-die design pays a consistent price. Tier medians stand in for chips without rows and are labelled “estimate — unmeasured”: base 0.94 / 5.8 ms, Pro 0.95 / 5.8 ms, Max 0.91 / 5.1 ms, Ultra 0.83 / 5.2 ms; M6 and M6 Pro take the base and Pro medians. The M5 Ultra is the one explicit assumption: 85 % of its 1,200 GB/s (the Ultra tier’s share) and 1.5 ms of fixed cost (between the M5 Pro’s 2.6 and the M5 Max’s 0.5), until a measured row exists. It still has no rows of the table the profiles are fitted on (the F16, Q8_0 and Q4_0 reads of one 7B model), but the first measured runs land close to it: Tom’s Hardware’s llama.cpp run of Qwen 3.8 27B at Q4_K_M writes 46.3 tok/s with an empty context and 30.1 at 131K against the profile’s 44.8 and 30.5, and oMLX runs on the 64-core chip write 53.5 at 4K and 41.3 at 64K against 56.9 and 43.4. The M6 Mac mini, on the base-tier medians, writes 9.6 at 4K and 7.1 at 64K in oMLX runs against the tool’s 9.3 and 7.4. The chip name is matched without its GPU bin (M5 Max (40-core GPU) → M5 Max); a platform without a profile (AMD ROCm boxes) gets no speed number rather than an invented one.

Architecture cost is the extra fixed time a model’s design adds per token, on top of the chip’s own. The class comes from the config: a dense model adds nothing; a mixture of experts adds its routing; one with linear-attention layers (the Qwen 3.5, 3.6 and 3.8 mixtures, Nemotron 3) adds those layers’ recurrent state; one with compressed (latent) attention (GLM-5.x, Kimi, DeepSeek-R1) adds the decompression; DeepSeek V4 is its own class, set by hand in the overrides file, because its FP4 experts and 512-wide attention measure far above the others. Each cost is the median over the measured GGUF rows of its class at short context, after the read and the chip’s fixed cost are subtracted:

Qwen 3.8 Flash Next has been its own class since 0.16, read from its model type (qwen4_exp): three of its files on llama.cpp on an M5 Max, measured by a rival engine’s maintainers, cost 10.1 to 11.5 ms per token beyond the read, twice the other hybrids, from its 512 experts and the n-gram lookup each token makes. Until then it was costed as a hybrid and came out 20 to 50 % fast.

Read factor. On Apple silicon, GGUF K-quant and IQ files (Q4_K_M, UD-Q4_K_XL, IQ2_M …) read about 25 % slower than the plain Q4_0, Q8_0, F16 and MXFP4 formats, because their dequantization is heavier on Metal. One dense row measures it (1.25, kept as an assumption until more rows exist), and it applies only on Apple silicon: no CUDA row measures it, so the Spark reads K-quants at the plain rate. MLX files read at the plain rate.

Context cost. Beyond the cache re-read (already in GB per token), attending over a long context costs time that does not show up in the bytes: 5 ms per token per 32K of context for latent-attention models (an assumption; see section 8 on the rows that could have fitted it), nothing for grouped-query attention, and on MLX a cost that depends on the chip generation: 6.3 ms per 32K on the M1 to M4 chips (fitted on 14 rows of the M3 Ultra and M1 Ultra sweeps to 128K) and 0.7 ms per 32K on the M5 generation (fitted on 7 rows of the oMLX M5 Max ladders to 128K; section 6.2). Because the cache is also in the read, speed visibly falls as the context grows: more for classic-attention models, less for hybrids, and more on the M1 to M4 chips than on the M5.

Sparse attention (0.17.3). Qwen 3.8 Flash Next’s attention layers do not read the whole context. Its config carries an index (indexer_budget = 2048): before each attention layer, a small scorer looks over every cached token and picks 2,048, and only those are read. On MLX, where oMLX runs that index, the tool charges the cache read and the attention cost for those 2,048 tokens, plus an index cost per 32K of the whole context. The index cost is fitted on the oMLX M5 Max rows past 8K, like the attention cost, and comes out at zero (3 rows): with a Flash Next token counted at the 6.67B parameters it reads (section 8, item 15), the read over 2,048 cached tokens and the fixed cost account for the whole time at 32K, 64K and 128K. The term stays in the formula for the next sparse model whose rows need it; the M1 to M4 chips, which have no such row, take the M5 value scaled by the gap between the two generations’ MLX attention costs, also zero, labelled assumed. The rule is keyed on the model type in constants.ts › SPARSE_ATTENTION, like the n-gram table (section 5), so the live records get it without an export. Without it, Flash Next on an M5 Max read the whole 128K cache like its sibling Qwen3-Coder-Next, which has the same 12 attention layers out of 48 and no index, and came out 23 to 39 % slow on the oQ4e build from 32K to 128K (39 tok/s at 128K where oMLX measures 64). With it, the M5 Max estimate holds at 59.4 tok/s at every length, against 68.9 measured at 4K, 68.5 at 32K (14 runs), 63.5 at 64K (15 runs) and 64.0 at 128K (4 runs): 7 to 14 % low at every length, which is the level of the fixed cost (section 6.6), not the context. On the mlx-community 4-bit file the M5 Max goes from 52 and 38 tok/s at 32K and 128K to 58 at both; the M4 Max from 35 and 19 to 48. llama.cpp files are still charged the whole context: no long-context llama.cpp run shows that engine using the index, and the cell says so. GLM-5.x, DeepSeek V4 and MiniMax M3 have indexes of their own (2,048 tokens, 512 tokens and 16 blocks of 128), but no long-context run to check them against, so they keep the whole-context charge (section 8, item 30).

Why not a term scaled by the share of attention layers. The cache read already counts only the layers that keep a cache (Flash Next’s 12 of 48), so a hybrid is not charged for its linear layers. Flash Next shares its share of attention layers, a quarter, with Qwen3-Coder-Next and Qwen 3.8 27B, and those two write within 8 % of their oMLX ladders at every length on the M5 Max (the 27B to 64K, Coder-Next to 128K, Nemotron 3 Super at 32K; Coder-Next’s held-out M5 Pro ladder sits 3 to 10 % low at every length, short ones too). A term scaled by that share would leave all three, and Flash Next, where they are. Fitted on the M1 to M4 rows, it does fix Qwen3.5-397B at 128K on the M3 Ultra (from 24 % slow to 4 % fast) but moves dense Qwen 32B from 1 to 6 % fast to 7 to 12 % slow and Mixtral from 19 to 25 % slow to 28 to 38 % slow, so the median error of the long-context rows gets worse (1.052 to 1.075). What separates Flash Next is the index, not the layer count.

How the fit is judged. The terms above are fitted on the training split of validation.json (137 measured runs, each row with its source, grade and context length) by coordinate descent on medians, and judged on a held-out split the fit never saw: every Gemma 4 row, the omlx benchmark site’s short-context submissions, since 0.16 the oMLX community runs on chips outside the 128 GB comparison (the M5 Pro, the M3 Ultra, the 64-core M1 Ultra, the M5 Ultra), and since 0.16.1 one whole model family, Qwen3.6-35B-A3B, on every chip and GPU bin the oMLX community ran it on, split by submission so that a contributor’s sweep is never half in each. On the 64 held-out rows the median error is a factor of 1.13 (90th percentile 1.41); the multiplier model scored 2.59 on the same rows. The 23 rows held out in 0.16 score 1.11; the 41 added in 0.16.1 score 1.13 (1.24 before the expert-layer rule), the hardest being Qwen3.6-35B-A3B (1.27: it writes 10 to 45 % faster on MLX than the tool says; section 8, item 27). On the training rows: 1.07 against 1.98. The release rule was a held-out median under 1.25 and no measured row getting worse than the multiplier by more than 1.30; both held. factors.json carries every term with its row count, range and source, and calibrate.py --report regenerates the row-by-row comparison. Chips whose profile is a tier median are flagged in the sheet.

6.2 Runtime: GGUF (llama.cpp / LM Studio) vs MLX

The baseline above is llama.cpp with a GGUF file. Apple’s MLX runtime (MLX files, used by LM Studio’s MLX engine, Ollama’s MLX backend, mlx-lm) changes the fixed cost per token, not the read: it trims the fixed cost of the bigger mixtures of experts, makes no difference on dense models (8-bit files and big dense models: 0.96 to 1.2×), and on small mixtures (under 6B active) it sets a fixed cost of its own that has nothing to do with the chip’s llama.cpp one. So the tool multiplies the fixed cost by a factor per model group for dense models and the bigger mixtures, fitted on the MLX rows of the training split, and charges small mixtures by their expert layers (below):

Small mixtures of experts on MLX (0.17). For a mixture under 6B active (gpt-oss-20b and 120b, Qwen3-Coder-Next, Qwen 3.6 35B-A3B, Gemma 4 26B-A4B, Qwen3-30B-A3B) the fixed cost per token is the model’s number of expert layers times a cost per expert layer that depends only on the chip generation:

fixed (MLX, under 6B active) = expert layers × cost per expert layer [generation]

Each value is the median, over the short-context MLX rows of its generation in the training split, of the measured time per token less the read and the context cost, divided by the model’s expert layers (oMLX community runs of gpt-oss-20b, gpt-oss-120b and Qwen3-Coder-Next on every chip and GPU bin with runs, and the older mlx-lm rows). A generation with fewer than two rows of its own (the M6 has one) takes the previous generation’s value; one newer than every row takes the newest; a chip whose generation cannot be read takes the M1’s, the slowest. The expert layers are the config’s count of layers that carry experts (36 for gpt-oss-120b, 24 for gpt-oss-20b, 48 for Qwen3-Coder-Next). A model without a layer count falls back to the group factors of 0.16 (×1.07 on the M4 and newer, ×1.18 on the M1 to M3), which factors.json still carries for that case.

Why a cost per layer and per generation: on MLX, every expert layer runs its router, its top-k and its expert kernels one after another, and that time does not shrink with a faster memory or a smaller file. On the M5 Max gpt-oss-20b, gpt-oss-120b, Qwen3-Coder-Next and Flash Next all cost 0.14 to 0.16 ms per expert layer, while the M5 Max’s fixed cost from the llama.cpp table, which earlier versions scaled, is half a millisecond for the whole token. The GPU bin is left out on purpose: a term for GPU cores (a cost per expert layer divided by the cores) fitted the small mixtures in the training rows better (1.04 against 1.07) and did worse on the held-out ones (the Qwen3.6 rows 1.40 against 1.29, the M5 Ultra 1.35 against 1.23, the 0.16 held-out rows 1.09 against 1.05, Gemma 4 level), because it learned how base and Pro chips run gpt-oss-20b and then made the 64- and 80-core Ultras too fast. Runs from the same weeks put a smaller GPU bin 0 to 10 % behind on mixtures (section 8, item 28).

Also: MLX files are 5–13 % smaller for the same nominal quant (the tool uses the actual MLX file size when you pick MLX), and MLX reads a prompt about 30 % faster than llama.cpp’s plain files on the M3 and newer chips and about a fifth slower on the M1 and M2 (section 6.4). Long context: on top of the cache re-read, MLX pays a cost per token per 32K of context that depends on the chip generation. On the M1 to M4 chips it is about 6.9 ms (fitted on the Qwen 32B and Qwen3.5-397B sweeps on the M3 Ultra and the Qwen 27B rows on the M1 Ultra, to 128K), which is why llama.cpp with flash attention catches up and passes MLX there somewhere past 30K tokens (on an M3 Ultra, MiniMax-M2.1 at 146K: MLX 6 tok/s against llama.cpp 12). On the M5 generation it is about 0.9 ms (fitted on the oMLX M5 Max ladders of Qwen 3.8 27B to 64K, Nemotron 3 Super at 32K and Qwen3-Coder-Next to 128K, accelerations off): the same model that loses 55 % from 4K to 64K on the M3 Ultra loses 23 % on the M5 Max, and the difference beyond the cache read is what the M5 GPU’s neural accelerators remove. A chip whose generation the tool cannot read (a custom machine) is charged the pre-M5 cost. The M5 Ultra inherits the M5 value unmeasured, like the rest of its profile. The crossover falls out of the terms instead of the hard-coded 24K–36K fade of earlier versions. A sparse-attention model pays these costs over its budget, not the context (section 6.1), which is why Flash Next on MLX barely slows to 256K while a llama.cpp file of it does.

6.3 Acceleration: MTP and draft models (DFlash / DSpark)

Some models can write several tokens per step. Two ways: MTP (multi-token-prediction heads built into the model — we detect them in the config) and draft models (a small helper model proposes a block of tokens the big model verifies in one pass; DFlash and DSpark drafts exist for many popular models — we find them on Hugging Face daily). Output is identical; only speed changes. The tool offers these only for models that actually have them, and shows the result as a range beside the baseline, never as the headline:

These ranges are computed by calibrate.py from the paired rows in benchmarks.json (each accelerated row records the unaccelerated speed of the same setup), so they move as measurements are added.

Gains depend on the task (code and math accept longer drafts than chat) and on quant (8-bit targets accept more than 4-bit).

6.4 Reading speed (prefill) and the wait before the first word

Before the first word the model reads the whole prompt: the system prompt, the document you pasted, the conversation so far. That pass is prefill, or prompt processing, and it is a different job from writing. Writing produces one token per step and is limited by memory bandwidth (6.1). Prefill pushes hundreds of tokens through the GPU in one batch and is limited by compute. Its number is a rate in prompt tokens per second, and the number you feel is the wait before the first word: the prompt length over that rate, made worse by the context already read.

prefill tok/s at a short prompt = K × kquant × class × mlx ÷ active parameters (billions)
K                               = GPU cores × per-core rate of the generation         (Apple silicon)
                                = the chip's constant                                 (the DGX Spark)
rate at context depth d         = short-prompt rate ÷ (1 + d ÷ D0)
D0                              = the runtime's halving depth × 0.25 ÷ the model's share of attention layers
                                  (a model's own published sweep, where it has one, on that runtime)
wait before the first word (n)  = (n + n² ÷ 2·D0) ÷ short-prompt rate                 for a prompt of n tokens

The constant is per GPU core of a generation, and it is remarkably steady. On the community llama.cpp table (the same rows as 6.1, which also carry pp512, prompt processing at 512 tokens), tokens per second times the model’s active parameters, divided by GPU cores, is the same number across every bin of a generation: M1 112 (99 to 127, 13 rows), M2 122 (110 to 136, 12 rows), M3 128 (124 to 134, 13 rows), M4 151 (148 to 157, 11 rows), M5 531 (482 to 546, 8 rows). The M1 Max 24-core reads at 0.755 of the 32-core, the M4 Max 32-core at 0.8 of the 40-core, the Ultras at 1.85 to 1.94× their Max. The file’s bits barely matter: Q4_0, Q8_0 and F16 prefill within about 10 % of each other on every chip, because the work is compute, not reading. The M5 generation’s per-core rate is 3.5× the M4’s: the neural accelerator in every GPU core, which is why Apple’s own MLX figures put the M5’s time to first token at 3.3 to 4.1× the M4’s while its writing speed gains 1.2×. The DGX Spark, whose cores mean nothing here, carries one constant (21341, 2 rows). Chips without rows take their generation’s rate on their own core count (the M5 Ultra, the 32-core M5 Max) and say so; a generation without rows (the M6) takes the newest one’s, flagged as assumed. Memory size changes nothing; the CPU changes nothing. Current llama.cpp builds read faster than the table’s rows on the older chips and a little slower on the newest: fitted on Tom’s Hardware’s ladders of Qwen 3.8 27B at Q4_K_M, the M4 Max reads 246 tok/s at a short prompt against the tool’s 191 (1.29 times), the DGX Spark 767 against 676 (1.13), and the 80-core M5 Ultra 1,155 against 1,346 (0.86); an M4 Mac mini reads a 9B model at about 1.4 times its constant (Alex Ziskind’s review). The constants stay on the table until current-build rows exist for every generation (section 8).

Which GPU the number is for. Where one machine row covers two GPU bins on the same bandwidth (the Mac mini M5 Pro with a 16-core or 20-core GPU, the M5 Ultra with 64 or 80 cores), writing speed is the same on both and reading speed is not, so the reading estimate is for the bin the row’s list price buys at that memory size: the smaller one, unless the size only comes with the larger (a 64 GB MacBook Pro M5 Pro is 20-core, a 512 GB M5 Ultra 80-core). Rows without a price follow the same rule, the base configuration. The sheet names that bin next to the price and next to the reading speed, and adds one line for the larger bin: how much faster it reads (in proportion to its cores: 20 over 16 is 25 %), its short-prompt rate and 32K wait, and, on current machines, what Apple charges for it at that size (+$200 on the M5 Pro Mac mini and MacBook Pro, +$100 on the 16 GB MacBook Air, +$200 on the iMac, +$1,300 on the M5 Ultra Studio). The Reading tab’s header says the same in brackets. Until 0.13.2 the estimate used the larger bin while the price was the smaller one’s.

Terms that move it, each fitted on measured rows with a prefill figure against the table’s constant for the same chip and cores:

• K-quant and IQ GGUF files: 0.88 of the plain-file rate (7 dense rows on an M2 Max; range 0.85 to 0.98).
• Architecture class: a mixture of experts prefills below its active-parameter share, 0.68 (range 0.46 to 0.92, 6 rows); one with linear-attention layers 0.63 (3 rows); Qwen 3.8 Flash Next is its own class at 0.34 (one published sweep on an M5 Max: its linear layers and engram lookups); these are ratios at a 512-token prompt (mac-llm-bench, MIT). Latent-attention mixtures (GLM-5.x, Kimi, DeepSeek) read at 0.48, from MacStories’ GLM-5.3 Flash runs on MLX at 3,408 to 258,025 tokens against the dense MLX rate of the same chip (0.36 on the M5 Ultra, 0.61 on the M3 Ultra); DeepSeek V4 takes that value, flagged as assumed.
• MLX: fitted on MLX runs at real prompt lengths, 4K tokens and more: the oMLX community benchmark database (omlx.ai; the 568 most recent runs of Qwen 3.8 27B in the plain 4-bit MLX file on 14 Apple chips as of 2026-10-14, one ladder per machine and memory size), MacStories’ M5 Ultra review and the mlx-serve release notes. Against llama.cpp’s plain-file constant for the same chip and cores, MLX reads at 0.80 on the M1 and M2 (16 machines, 0.74 to 0.88), 1.27 on the M3 and M4 (18 machines, 1.09 to 1.32) and 1.31 on the M5 generation and newer (13 machines, 1.09 to 2.13). The M3 GPU is where MLX’s kernels pull ahead per core; on the M5 the neural accelerators lift both runtimes alike. A chip whose generation cannot be read takes the M1 and M2 value. MLX runs at a 512-token prompt do not set the factor, because the per-request start-up cost weighs on a short prompt (oMLX’s 1K runs read at 0.81 to 0.99 of its 4K ones, the fastest chips losing most). Until 0.15 the factor was 0.61 on the M5 and 0.65 before it, from such runs: six mac-llm-bench rows measured in April 2026 on a fanless MacBook Air M5 with mlx-lm 0.31.2 (the same machine’s llama.cpp runs write at under half the speed its MLX runs do) and one M2 Max row. They stay in the data, marked excluded. oMLX 0.7’s experimental INT8-activation prefill (+34 % at a 32K prompt on an M5 Max, off by default, can change outputs) is not in the factor.
• The context already read: attention over the context costs more with every token, so the rate falls as the prompt grows, by the depth D0 at which it has halved. It is fitted per runtime on measured ladders, each the wait before the first word for prompts of several lengths read from an empty context: wait = (n + n² ÷ 2·D0) ÷ r0, least squares on the prompts of 4K and more, which matches every point within 6 %. On llama.cpp D0 = 81,781 tokens, the median of Tom’s Hardware’s three ladders of Qwen 3.8 27B at Q4_K_M (a 2,048-token prompt on top of 0 to 131,072 tokens of context: the M5 Ultra 42,621, the M4 Max 81,781, the DGX Spark 185,009). On MLX D0 = 95,613 tokens, the median of the 11 oMLX ladders whose prompts span at least eight-fold (44,291 to 120,239); a ladder spanning less, or a single length, takes its short-prompt rate from this value. Both are for a model with attention in a quarter of its layers, as Qwen 3.8 27B has; other models scale D0 by their share of full-attention layers (a model with attention in every layer halves four times sooner), an assumption the sheet states. Qwen 3.8 Flash Next keeps the halving depth of its own published llama.cpp sweep (a 128 GB M5 Max: 966 tok/s at an empty context, 324 at 32K, 80 at 256K; D0 = 19,257), which reproduces the sweep author’s “28 minutes to fill 262K”; on MLX it takes the fitted value. Until 0.15 that one sweep set D0 for every model and runtime, and long prompts on llama.cpp waited 1.5 to 3.7 times longer in the tool than Tom’s Hardware measured at 131K.

What the wait is for. Every wait is for a prompt read in one go: the 8K, 32K and 128K sizes, and the column’s whole context pasted at once. In a chat the runtime keeps what it has read (llama.cpp’s prompt cache, which LM Studio and Ollama sit on; mlx_lm.cache_prompt and the chat cache in MLX), so each turn waits only for its new text, and a session that grows to 256K pays that read in pieces over its life, never at once. The whole-context figure is the cost of dropping a codebase or a book in at one stroke, and on a 27B dense model at 256K it is about 17 minutes on an M5 Max with llama.cpp and 10 with MLX (Flash Next’s own sweep, whose attention layers slow it far sooner, says 28 minutes on llama.cpp). That is the physics of reading two and a half novels on a laptop, not a per-message cost. The Reading tab opens on the 8K prompt, the everyday size; the sheet lists the others.

Linked machines (0.13). On a layer split the machines read the prompt in turn, so the pool reads at one machine’s rate (measured flat: +4.6 % on two Sparks, +17 % with llama.cpp’s later batching). Under tensor parallel the prompt work is divided, so the rate should scale with the count; only a short-prompt point exists (time to the first word 233 ms on one Spark, 149 ms on two, 1.56×), and the tool uses count ^ 0.75 (1.68× at two, 2.83× at four), marked assumed.

What the tool shows. “How fast?” has a Reading tab beside Writing: for one machine, the wait before the first word for each model at a prompt size you pick (2K, 8K, 32K, 128K, or the column’s own context), banded instant (under 3 s), short wait (to 15 s), get a coffee (to 90 s), painful. The sheet’s “Reading speed” section carries the short-prompt rate, the waits for 8K, 32K and 128K, and “how this number is made” with the constant, the cores, and every factor with its source. A build that streams from the SSD has no reading estimate. Every prefill row, ladder and sweep is in benchmarks.json with its source (a ladder point names each benchmark submission it is the median of); calibrate.py derives the terms and speed_model.py mirrors the formula, pinned by prefill-cases.json on both sides.

Worked numbers, Qwen 3.8 27B at Q4_K_M: the 48 GB Mac mini M5 Pro at its $2,299 list price (16 cores) about 270 tok/s at a short prompt, a 32K prompt waits about 2.4 minutes; with the 20-core GPU (+$200) about 340 tok/s and 1.9 minutes; Mac Studio M5 Max (40 cores) about 670 tok/s, 1 minute; an M4 Pro mini at 16 cores about 77 tok/s, 8.6 minutes (96 tok/s and 6.9 minutes at 20). In the 4-bit MLX file the same machines read at about 400, 500, 1,000, 110 and 140 tok/s, and the 32K waits are 1.6 minutes, 1.3 minutes, 38 seconds, 5.8 minutes and 4.6 minutes; oMLX measured 419 tok/s over a 4K prompt on the 16-core M5 Pro, 508 on the 20-core, 935 to 1,022 on the M5 Max and 106 and 132 on the M4 Pros. The mini writes comfortably and still makes you wait for a long document, and the M5 over M4 decision is about reading, not writing.

6.5 “Feels like” — the translation next to every speed

6.6 Every way to get 128 GB

The seventh tile puts one model at one build on every route to about 128 GB of memory for local AI, the comparison the “Every way to get 128GB” video makes: a gaming PC with 64 or 128 GB of DDR5 and the model’s experts in system memory, Strix Halo mini PCs (Framework, GMKtec, Minisforum), used Mac Studios (M1 Ultra, M2 Ultra, M4 Max), the Mac Studio and MacBook Pro M5 Max, the Surface Laptop Ultra (RTX Spark), the DGX Spark and the ASUS GX10, two linked 64 GB Sparks, the 256 GB M5 Ultra (there is no 128 GB one), four-card PCs (MI50, R9700) and an RTX PRO 6000 with system memory. For each it shows the price, the memory speed, the memory the AI can use, whether the model fits, the writing speed with the time for a 1,000-word answer (about 1,330 tokens), and the time to read a 50-page document (32,000 tokens) where a run measured it. What the tool does not compute sits in one hand-curated file, public/data/llm-sizer/routes-128gb.json: every price with its date, grade and store or tracker, the memory speed and the usable memory with their sources, and every published measurement with its engine, build, date and links. Prices move weekly; refreshing them means editing that file and nothing else. /tools/llm-sizer?view=128gb opens the tile.

Which file a route runs. At the chosen build a route runs the file every other view uses when it fits (MLX on a Mac, GGUF elsewhere), else the largest smaller file of the same build that fits with 10 % to spare, else the largest that fits at all, the way the fix search steps down (section 5). Two routes can therefore run different files of one build: Qwen 3.8 Flash Next at Q4 runs the 111.5 GB MLX 4-bit file on a Mac and the 100.6 GB Q4_0 on Strix Halo and the Spark. Flash Next’s n-gram table (51 of its 180B parameters) stays out of memory where the engines allow it: on a Mac with an MLX file (oMLX keeps it on the SSD, so the 4-bit file needs about 80 GB, which is what oMLX’s peaks of 69 to 79 GiB say; the same rule as every other view, section 5) and on the RTX PRO 6000 (vLLM keeps it in system memory, as the published run does). Plain llama.cpp files keep it inside the weights and need the whole file. The usable memory is the engine’s for the Macs and the linked Sparks (sections 4 and 7; a test holds them equal) and the published figure elsewhere: about 120 GB on Strix Halo under Linux and 96 on Windows, about 120 on the Spark, 64 GB by default on the Surface and 96 after a setting change, the card’s 96 GB plus the PC’s memory on the RTX PRO 6000.

How the writing speed is made. One rule per kind of route:

• Catalog machines (the Macs and the Spark): the engine’s own estimate, the fixed-cost model of section 6.1 with the chip’s profile, MLX on the Macs. A test holds it equal to the estimate every other view shows.
• Two linked Sparks: the engine’s layer split (section 6.1), one Spark’s time for the whole model plus the fitted 3.9 ms hop.
• Routes with a profile of their own (Strix Halo, the RTX PRO 6000): the straight line of milliseconds per token against gigabytes read that the chip profiles use, fitted on published rows of other models with the engine’s architecture costs taken off first. The comparison’s own three models are never in those rows; they are the check.
• No calibrated rule (a 12 to 16 GB card with the experts in system memory, the RTX Spark laptop, four-card PCs): no estimate, and the row says why. No plain run of these models on a gaming card with system memory is published, so the share of the PC’s memory speed the experts reach is unknown; the row shows instead the ceiling that memory speed alone sets (the weights on the card at the card’s speed, the experts in system memory at about 90 GB/s, nothing else costing time), labelled as a ceiling. Splitting a model across four cards adds memory, not speed, and no four-card run of these models exists.

A published run wins. Where a plain run of the model at the build is published for a route, the row shows it and the estimate moves to the detail. Where the two differ by more than 15 %, or the run used another runtime than the route’s estimate (vLLM against a llama.cpp profile), the detail gives the reason from the data file, and a test fails if any such row has none. Speed-up numbers (MTP, draft models, Strata’s speculative decoding) are their own line and never stand in for a plain one. A plain run too thin or too contradictory to stand (the M5 Ultra’s) is kept in the detail and the estimate stays the headline. On a route where a run of the same kind of model missed, an estimate for any other model carries a “less sure here” note.

Reading speed is not computed here: it follows the chip’s compute and the engine, not the memory speed (section 6.4), and on the DGX Spark and the RTX PRO 6000 it is most of what the extra money buys. The column shows the 50-page time only where a run measured reading near 32K, with the engine named, and “not measured” everywhere else.

The calibration table. Every published plain run in the file against the tool’s estimate for the same build, at the run’s own context, with the bits over the model’s total parameters as the catalog counts them. npx tsx scripts/llm-sizer/routes-table.ts regenerates it from the data file; routes.test.ts pins the rows the text below relies on.

What the table says:

• The Macs land within 15 % on Flash Next and Qwen 3.8 27B, except the 48-core M1 Ultra and Flash Next on the M2 Ultra. It has 800 GB/s on paper, as much as the M2 Ultra, and writes Flash Next at about 36 where the M2 Ultra writes 64. Its gpt-oss-120b runs (about 40) date from March and April; the 64-core chip wrote 42 to 48 in the same weeks and 53 in October, because oMLX got faster in between. Runs from the same weeks put the 48-core GPU 5 to 10 % behind the 64-core on mixtures (section 8, item 28), and the expert-layer rule (section 6.2), which has no term for the GPU bin, estimates 57 for both, 7 % above the 64-core’s October runs. Flash Next, at 6.67B active, is outside the rule and keeps the group factor, which reads this chip 39 % high.
• gpt-oss-120b lands within 8 % on the M5 Max, the M4 Max and the M2 Ultra since the expert-layer rule (it read 46 % high on the M5 Max and 20 % on the M2 Ultra in 0.16.1). It still reads high on the M5 Ultra: +18 % against the heavier mxfp4-bf16 file, which keeps the attention and embeddings in bf16, and +27 % against the one run of the M5 Max’s MXFP4-Q8 file (90 at 8K). The M5 Ultra’s profile is an assumption (section 6.1), and its runs suggest the Ultra pays more per expert layer than the M5 Max.
• The Spark’s and the RTX PRO 6000’s Flash Next runs are vLLM. The Spark’s plain run, about 20 tok/s from one forum post, sits far under the llama.cpp estimate (36), and nobody has published llama.cpp on the Spark for this model; on gpt-oss-120b, where llama.cpp’s own run exists, the Spark’s profile is within 10 %.
• Flash Next on the M2 Ultra reads 17 % under its run (53.6 against 64.4) since 0.17.1 counts the 6.67B parameters a Flash Next token reads, where it counted 6B (section 8, item 15). The fixed cost of the bigger mixtures on MLX is one factor fitted across six models, and this chip runs Flash Next faster than that factor says; the M5 Max reads 14 % under, the M4 Max 2 %.
• Strix Halo’s profile is within 10 % on gpt-oss-120b and the 27B and reads Flash Next 19 % high. The owner’s file is IQ4_XS, and the one other IQ4_XS row in the profile’s data (Nemotron 3 Super) misses the same way: IQ files read slower on that Vulkan backend than the tool has a term for.
• Two figures differ from the ones the video’s research first listed, both because of the build: the 48-core M1 Ultra writes about 25 on the heavier oQ4e-fp16-mtp copy (one session, 24.7 at 1K) and about 36 on the oQ4e-mtp copy the M5 Max’s 69 comes from; the M4 Max writes about 50 on that copy at 4K, and 60 on another 4-bit build at 32K. The table compares one build across the chips.

Routes the tool does not estimate honestly, and what it shows instead: the gaming PC with system memory (a ceiling, and Strata’s speed-up runs); the Surface Laptop Ultra and the four-card PCs (nothing: no run, no profile); the Spark on Flash Next and the 27B (vLLM’s measured number; the llama.cpp estimate in the detail); the 48-core M1 Ultra, Flash Next on the M2 Ultra, gpt-oss-120b on the M5 Ultra and the RTX PRO 6000 (the measured number); the two 64 GB Sparks (the engine’s estimate with a “less sure” note: no run before they go on sale); the M5 Ultra on Flash Next (the estimate: its plain runs read 42 to 124).

7. Other machines: the same math, different facts

One sentence to remember: memory adds up across cards and boxes; speed adds up only under tensor parallel on a fast link.

8. Known limitations (read before arguing with a number)

1. Everything speed-related is an estimate unless a measured row exists for exactly your chip, model, quant and runtime — then we show the measurement next to the estimate.

2. Fits ≠ runs well. A model that loads at 2-bit, or crawls at long context, is running the way a car runs in first gear. Rings tell you the compromise.

3. Sustained load is real. Long inference sessions run the GPU flat out; laptops get hot and may throttle. The tool doesn’t model thermals.

4. The macOS limit override is unsupported by Apple; the tool defaults to the official behavior.

5. Quantization quality varies by quantizer and method (dynamic “UD” quants keep more accuracy at the same size); we show quantizers’ own quality notes where published and don’t invent scores.

6. MoE active parameters are the parameters one generated token reads: the experts it is routed to, the weights every token passes through and the output layer. Makers’ published figures are used where they count that, the config-derived estimate is the fallback, and item 15 lists the exceptions.

7. Models newer than the data date aren’t in the tool yet; custom entries are the workaround.

8. “Architecture assumed conventional” entries use the pessimistic classic cache formula until someone tags the architecture.

9. Quick custom models (name, parameters, file size) get an assumed design: about 32 × (parameters ÷ 8B)^0.4 layers with 8 KV heads of 128, all caching. Enter the real fields in the full form for exact context math.

10. Two-point chip profiles. The M1, M3, M5 and the DGX Spark have only two dense rows, so their line has no slack and no error estimate; the Spark’s rows are the maintainers’ own, the Macs’ come from one community table.

11. Thin classes and one-row terms. The linear-attention and latent-attention mixture costs rest on two rows each, DeepSeek V4 on three (range 13 to 33 ms); the K-quant read factor rests on one row and is labelled assumed; the MLX cost per expert layer rests on 9 to 11 rows per generation and the M6 has a single row, so it borrows the M5’s (item 27); the latent-attention context cost is an assumption. These are the terms a single new measurement can move.

12. The held-out rows cover no latent-attention mixture and no mixture on the Spark, so the 1.14 median is a statement about dense models, Gemma’s mixtures and the Spark’s small dense models; the oMLX M5 Max submissions that run past 8K moved from the held-out set to the training set in 0.10, because they are the only long-context M5 rows there are, which leaves 13 held-out rows. Every row came from a public post; none was measured under controlled conditions.

13. Rows the model gets wrong by 2× or more. MiniMax-M2.1 at 30K and 146K of context (mlx-lm issue 763) runs at half the model’s prediction on both runtimes; Kimi K2.6 at 10K and 20K on a 512 GB Studio with a 440 GB build runs at a third and a fifth. Both look like memory pressure and are kept as report-only rows; a long-context term that catches them needs cleaner measurements.

14. Memory margins are policy, and their residuals are known. Against owners’ peak readings, with every reading converted to decimal GB where its unit is known (oMLX reports binary gigabytes), the tool over-estimates by 0.5 to 1.4 GB on 17 to 30 GB dense GGUF files, by 0.3 to 1.7 GB on a 27B MLX file from 4K to 64K, by 1 to 5 GB on a 224 to 422 GB MLX file, and under-estimates a 31B GGUF file at 37K by 2.4 GB. On Flash Next’s MLX files with the table on the SSD it over-estimates by 1.3 GB on the 111.5 GB file at 4K and by 3.9 GB on the 106.3 GB oQ4e build at 4K, rising to 10 GB at 64K and 20 GB at 128K: oMLX’s peak on that model grows by 5 GiB from 4K to 128K, while the MLX prompt-scratch margin adds 18 GB. A Flash Next verdict at long context is on the safe side. Readings whose unit or method is not stated are kept as published and flagged in validation.json. Read a tight verdict with that in mind.

15. Active parameters are the makers’ counts, checked against the files. A mixture of experts’ speed depends on the parameters one generated token reads: the experts it is routed to, the weights every token passes through and the output layer, which is read in full for every token. The input embedding (one row is looked up per token), n-gram tables, multi-token-prediction heads and vision towers are left out. Makers count in different ways, so a published figure is used where it agrees within 5 % with the count from the model’s own tensor table (read from the file headers on Hugging Face); OpenAI’s gpt-oss figures (3.61B and 5.13B) are exactly that count. Where it does not, the overrides file carries the tensor-table count and the reason. Qwen3-Coder-Next reads 3.56B per token, not the 3B of its name: the card rounds 3.25B down and leaves out the 0.31B output layer. Qwen 3.8 Flash Next reads 6.67B, not 6B: Qwen’s figure leaves out the 0.64B output layer, a different convention rather than a mistake, but the output layer is read for every token. GLM-5.2’s card publishes no count; the config-derived estimate (54.0B) counted the multi-token-prediction layer, the input embedding and the shared experts twice, and the tensor table gives 40.3B, in line with GLM-5’s published 40B for the same design. Measured reading speeds back the tensor counts: on the oMLX community runs at a 4K prompt, Qwen3-Coder-Next’s estimate on eight chips went from a median of 20 % too fast to 7 %, and GLM-5.2’s on the 80-core M3 Ultra from 49 % to 32 % too slow. So far seven mixtures are checked this way (gpt-oss-20b and 120b, Qwen3-Coder-Next, Qwen3.6-35B-A3B, Gemma 4 26B-A4B, Flash Next and GLM-5.2); the rest carry the published figure or the config-derived estimate unchecked, and that estimate has been wrong by 2× on models whose configs hide the expert width (MiniMax M3) or use a latent expert dimension (Kimi K3). Every validation row must use the catalog’s own count and cache bytes (a test enforces it), so a passing fit says something about the live path.

16. The M5-generation MLX context cost rests on seven rows of three models on one machine. The 0.7 ms per 32K comes from the oMLX M5 Max ladders (Qwen 3.8 27B at 16K, 32K and 64K, Nemotron 3 Super at 32K, Qwen3-Coder-Next at 32K, 64K and 128K), two submitters, accelerations off (Flash Next’s long-context rows fit its index cost instead, section 6.1), and it is applied to every M5 and later chip, including the M5 Ultra, which had no rows when it was fitted (its first oMLX runs write within 6 % of the tool at 4K to 64K; section 6.1). The pre-M5 value rests on the M3 Ultra and M1 Ultra alone. A long-context ladder on an M4 chip, or on an M5 Ultra once it ships, is the measurement that would settle which generation each value really belongs to.

17. The linked-machine terms rest on a handful of published series (section 6.1): four hop points on Macs from two series in the TCP era of Thunderbolt (RDMA pipeline hops measure about 0.2 ms, so a layer split over RDMA runs closer to one machine’s speed than the tool says), two on Sparks, four dense and four mixture points for tensor parallel, all on M3 Ultra, M4 Pro, M5 Max and Spark hardware. No three-machine tensor series exists; the GPU-box constants (1.0 ms hop, 1.5 ms per doubling) are assumed from one RTX PRO 6000 pair; the reading-speed exponent is assumed from one short-prompt point.

18. Dispatch-bound pools. A very deep mixture of experts in tensor parallel can run far below the bandwidth maths: Kimi K3 (92 layers, 16 experts per token) on four M3 Ultra Studios wrote 2 tok/s where the estimate says about 36, because per-kernel dispatch overhead dominates. The tool flags any mixture deeper than 60 layers under tensor parallel and keeps the number as a ceiling.

19. The Spark’s usable memory in a cluster is nearer 105 GB per machine under vLLM (page cache, memory utilisation 0.8 to 0.9) than the 129 GB the 8 GB reserve leaves; the largest model with a published two-Spark run is 177 GB, and a 204 GB one failed to load. Treat a two-Spark pool above about 210 GB as optimistic.

20. What linking needs that the tool cannot check: RDMA over Thunderbolt 5 needs macOS 26.2 or later, enabled from Recovery on every machine, a direct cable between every pair for tensor parallel (no Thunderbolt switch exists), identical macOS builds, and the wired limit raised on each; a Spark pool needs the 200 Gb/s cable (10 GbE keeps about 40 % of the speed) and NCCL actually on RDMA, which silently falls back to TCP. LM Studio and Ollama do not split a model across Macs, so a pool means llama.cpp, MLX, exo, vLLM or TensorRT-LLM. Sparks link only to Sparks (or an ASUS GX10), never to a Mac.

21. The default GPU limit is macOS 26’s, read from public logs. Apple documents none of the shares in section 4. Three sizes rest on one reading each (8, 18 and 36 GB), the two 256 GB readings do not name the macOS version (87 % only appears on macOS 26), and one 64 GB report on macOS 26 shows the old 75 %, probably set by hand. On macOS 15 the limit is lower (section 4), so a verdict near the edge can be too generous there; set 67 % or 75 % in Advanced to check.

22. Reading speed rests on one model. The MLX factors and both halving depths of section 6.4 are fitted on Qwen 3.8 27B, a dense model with attention in a quarter of its layers. Other models inherit them through their active parameters and their share of attention layers, and the class factors for mixtures of experts are still ratios measured at a 512-token prompt; how either holds at long prompts on other designs is an assumption. Where a class has been measured on two generations the factor differs: GLM-5.3 Flash reads at 0.36 of the dense rate on the M5 Ultra and 0.61 on the M3 Ultra, and MacStories’ Flash Next runs at about 0.33 and 0.59 (its class factor is 0.34), because the M5’s neural accelerators speed up dense layers more than expert layers. With one factor per class, mixtures of experts read high on the M5 generation and low on the older chips.

23. The halving depth differs by chip more than by runtime. On llama.cpp it is 42,621 tokens on the M5 Ultra, 81,781 on the M4 Max and 185,009 on the DGX Spark, and the tool uses the median: a 128K prompt on the M5 Ultra waits about 3 minutes in the tool against 4.9 measured, on the Spark 6 against 3.9. On MLX the M5 Max slows sooner than the other chips (its ladders give 44,291 and 28,653 against about 100,000 on the M5 Pro, the M6 and the M4 Max), so over a 32K prompt the tool’s M5 Max reads 855 tok/s on average where oMLX measured 684, and its maintainer 615 on the oQ4e file.

24. The 80-core M5 Ultra reads below its core count. Per core it reads about a tenth slower than the 64-core chip in the oMLX runs (1,664 tok/s over a 4K prompt, one run, against 1,500 on 64 cores) and in MacStories’ review (1,701 over a 6,091-token prompt), so its estimate is 14 to 18 % high. The tool scales reading with cores and has no term for it.

25. Current llama.cpp reads faster than the table on the older chips. The per-core constants come from the community table, whose rows for the older chips were measured with earlier builds: Tom’s Hardware’s M4 Max reads at 1.29 times its constant and an M4 Mac mini at about 1.4 times, the DGX Spark at 1.13, the M5 Ultra at 0.86 (section 6.4). The constants stay until current-build rows exist for every generation; MLX numbers do not carry this gap, because the MLX factor is fitted on measured MLX runs against the same constants.

26. The community runs are self-reported. The oMLX database records what each submitter’s Mac did, with no control over background load, heat or memory pressure. The tool keeps a run only when its writing speed is within 10 % of the chip’s usual speed at that prompt length, and names every kept and dropped submission in benchmarks.json.

27. Small mixtures of experts on MLX: what the expert-layer rule leaves. Since 0.17 a mixture under 6B active costs its expert layers times a cost per layer for the chip generation on MLX (section 6.2); until then the chip’s fixed cost from the llama.cpp table was scaled, which read gpt-oss-120b 46 % high on the M5 Max. What the rule still misses: one rate per generation does not fit every model. Qwen3.6-35B-A3B costs well under what Qwen3-Coder-Next costs per expert layer on the same chips (about two thirds on the M5 Max), so it writes 10 to 45 % faster than the tool says (a held-out family). gpt-oss-20b reads high on the base and Pro chips (+42 % on the 10-core M5, +16 % on the M4, +19 % on the 16-core M5 Pro): its MXFP4-Q8 files keep the attention and embeddings in 8-bit, so each token reads more than the bits over the total parameters say, and the slower the memory the more that shows. The M5 Ultra’s rows (held out) read 3 to 27 % high. Flash Next, at 6.67B active, keeps the group factor, as do the bigger mixtures.

28. GPU bins: a smaller GPU writes mixtures 0 to 10 % slower on MLX, more on the M1. Runs from the same weeks put the 48-core M1 Ultra 5 to 10 % behind the 64-core on gpt-oss-20b and Qwen3-Coder-Next, the 24-core M1 Max 14 % behind the 32-core on gpt-oss-20b, the 8-core M2 12 % behind the 10-core, the 60-core M3 Ultra level with the 80-core on gpt-oss-120b, 9 % behind on gpt-oss-20b and ahead on Qwen3-Coder-Next, and the M4 Pro and M5 Pro bins within 4 % of each other. On dense models the M1 bins differ more (Qwen 3.8 27B: 17 % on the M1 Ultra, 20 % on the M1 Max). The gaps 0.16 reported (a quarter on the M1 Ultra, 43 % on the M3 Ultra’s Flash Next) came from comparing runs months apart and from single sessions that other sessions on the same machine contradict (an 80-core M3 Ultra also wrote Flash Next at 28.7). The tool has no term for the GPU bin: one fitted on these rows (section 6.2) made the held-out estimates worse. The chip profiles are fitted per chip, not per GPU bin; the smaller bins’ rows are in validation.json with their GPU cores.

29. The 128 GB comparison’s own profiles are thin. Strix Halo’s rests on one owner’s eight Vulkan rows (worst row 36 % off: an IQ4_XS file), the RTX PRO 6000’s on two small models in llama.cpp’s own threads (the line runs through zero). They are checked on the comparison’s models where runs exist (section 6.6) and say so in the sheet; neither machine is in the catalog.

30. Sparse attention rests on one model on one chip. The index cost is fitted on three medians of the oMLX M5 Max runs of Flash Next (the oQ4e build, 13 to 18 October) and comes out at zero, and medians across sessions hide part of the slope: within one session the M5 Max writes 0 to 19 % slower at 128K than at 4K or 8K (four sessions, about 11 % in the middle; a 64-core M5 Ultra session on the 6-bit build goes from 76.2 at 4K to 67.5 at 128K), where the tool says none. The M1 to M4 value is assumed. GLM-5.x, DeepSeek V4 and MiniMax M3 carry indexes too and keep the whole-context charge until a long-context run of one of them exists; a llama.cpp file of Flash Next is charged the whole context. Reading speed has the same blind spot: in one M5 Max session oMLX reads Flash Next at 2,499 tok/s over a 128K prompt against 2,788 over 4K, while the MLX halving depth of section 6.4 (95,613 tokens at a quarter of the layers attending) has a 128K prompt read at about 60 % of the 4K rate, so the tool’s 128K wait for this model on MLX is about 1.5 times too long.

9. Worked examples

All numbers below come from the live data (file sizes read from Hugging Face, factors from factors.json) and are reproduced by the engine’s tests.

A. 32 GB Mac mini (M6), Qwen 3.8 27B at Q4 (LM Studio’s Q4_K_M, 16.8 GB), 32K context. The machine has 34.4 GB in file-size units. Available with the default GPU limit = min(34.4 × 78 %, 34.4 − 6 − 0) = 26.8 GB. Needed = 16.8 (weights) + 2.2 (cache: 64 KB/token at FP16 × 32K, plus the hybrid layers’ constant state) + 1.7 (buffers: 1.5 GB + 1 %) = 20.7 GB → runs at 77 %, a gold sphere. Speed ≈ 6.5 tok/s (“usable”; the M6 has no measured row yet, so it uses the base-tier profile and carries the estimate label). Turn the work toggle on: available = min(26.8, 34.4 − 6 − 16) = 12.4 GB → the ring says “runs if you close your apps”; the tooltip’s alternative is a 2-bit build that fits with the apps open.

B. 256 GB Mac Studio (M5 Ultra), GLM-5.3 Flash at Q4 (UD-Q4_K_XL, 199.7 GB), 128K context. Available with the default GPU limit = 87 % of 274.9 GB = 239.1 GB; needed = 199.7 + 1.5 (cache — compressed attention on 11 of 45 layers, FP16) + 3.5 (buffers) = 204.8 GB → runs at 86 %, a gold sphere. With the override, available = 274.9 − max(6, 15.1) = 259.8 GB → runs at 79 %; with the work toggle on as well (243.8 GB) it runs at 84 %. Speed on the M5 Ultra (assumed profile: 1,020 GB/s effective, 1.5 ms): 18B active × 4.97 effective bits ≈ 11.2 GB read per token at 8K → 13.8 ms read + 1.5 ms chip + 11.7 ms latent-attention mixture + 1.3 ms context ≈ 28 ms → ≈ 35 tok/s at 8K, ≈ 31 at 32K and ≈ 21 at 128K; with MLX the fixed part shrinks to 0.68× (≈ 41 tok/s at 8K); with its MTP heads, more again. The same model on a DGX Spark: 129.4 GB available after the 8 GB reserve; Q4 does not fit; the 3-bit build (UD-IQ3_XXS, 120.4 GB) fits tight at 96 %, at ≈ 15 tok/s (the Spark’s 258 GB/s effective and 2.6 ms, plus the same 11.7 ms architecture cost), and the 2-bit build (UD-Q2_K_XL, 108.7 GB) is the alternative at ≈ 16 tok/s.

C. 512 GB Mac Studio, GLM-5.2 at Q4 (UD-Q4_K_XL, 467.3 GB), 128K context, override on. Needed = 467.3 + 11.8 (cache — compressed attention, FP16) + 6.2 (buffers: 1.5 GB + 1 % of 467) = 485.2 GB; available = 549.8 − max(6, 30.2) = 519.5 GB → tight (93 %). It fits without the override too: macOS 26 holds back only 48 GB of a 512 GB Studio, so 498.2 GB is available and the file is tight at 97 %. The 3-bit build (UD-IQ3_S, 309 GB) would run comfortably.

D. 256 GB Mac Studio, GLM-5.2, 128K. A ring: with the override (259.8 GB available) the 2-bit build UD-IQ2_M (238.6 GB) fits with its cache and buffers (254.2 GB) at 98 %, tight; the alternatives are the smaller 2-bit UD-IQ2_XXS, also tight, and a 32K context. Without the override nothing fits, so the fix is the override plus the 2-bit build.

E. 512 GB Mac Studio, Kimi K3 (2.8T parameters). A dash: the smallest 2-bit build is 711 GB and needs about 724 GB, against 520 GB at most with the override. Its 1.3-bit build (466 GB) would load at 92 % with the override, tight — quality unknown, and only if you lower the floor in Advanced.

F. 64 GB MacBook Pro (M5 Max), Qwen 3.8 Flash Next. The machine has 68.7 GB; 81 % of it is 55.7 GB. Every regular 2-bit-and-up build starts at 75 GB, so none fits. Asked for Q4, the fix is the engram-on-SSD build: 45.8 GB in memory + 0.9 GB cache + 2.0 GB buffers = 48.6 GB → ring: “with the engram-on-SSD build (45.8 GB in memory, 39.1 GB streamed from the SSD)” (87 %). Asked for that build directly (IQ4_XS · engram on SSD in the column’s build list), the same 48.6 GB runs with no change, the bar shows 39.1 GB on the SSD beside it, and the speed reads “not estimated” with the publisher’s 36 tok/s on an M5 Max quoted as a note. The Q4_K_M · engram on SSD build keeps 54.5 GB in memory, 57.4 GB with its cache and buffers, over the 55.7 GB limit: on this machine it needs the override.

G. The buying map for Qwen 3.8 27B at Q4 (Q4_K_M), 32K, at least 12 tok/s. Every current configuration is evaluated as in A. With no minimum speed the cheapest that fits as asked is the 32 GB Mac mini (M6) at $1,299 (runs at 77 %, ≈ 6.5 tok/s, estimate); at 12 tok/s or more it misses on speed, and the pick becomes the 48 GB Mac mini (M5 Pro) at $2,299 (≈ 13 tok/s, comfortable, 19.5 GB to spare), with the 64 GB M5 Pro mini ($2,699, 35 GB to spare) as the alternative with headroom; a laptop preference makes it the 48 GB MacBook Pro (M5 Pro) at $3,099. Under a $2,000 budget at 12 tok/s nothing qualifies: the card names the 48 GB mini as $299 over the budget, the 32 GB mini as too slow (≈ 6.5 tok/s), and the 24 GB M5 Pro mini as qualifying only with a change (the memory-limit override and 8K context). The 512 GB Studio and the 64 GB DGX Spark fit at every setting but have no list price yet, so they are listed as qualifying without one and are never the pick.

H. Linked machines: GLM-5.3 Flash at Q4 (UD-Q4_K_XL), 32K, on two DGX Sparks and on two 128 GB M5 Max Studios. One Spark: Q4 does not fit (a ring at the 3-bit file, example B). Two Sparks on a layer split: the pool offers 2 × (137.4 − 8) − 4 = 254.9 GB; needed = 199.7 × 1.05 + 0.4 (cache) + 1.5 + 1.5 + 2.1 (buffers, one per machine plus 1 % of the weights) = 215.2 GB → runs at 84 %. Speed: one Spark’s time for the whole model is 64.1 ms per token (11.6 GB read at 258 GB/s effective, 2.6 ms chip, 11.7 ms latent-attention mixture, 5 ms context), plus one 3.9 ms hop → 68 ms → ≈ 15 tok/s; reading stays one Spark’s 501 tok/s. In tensor parallel: needed = 199.7 × 1.08 + 0.9 (the cache duplicated) + 5.2 = 221.7 GB → runs at 87 %; speed 64.1 ÷ 2 + 10.9 × log2(2) = 43 ms → ≈ 23 tok/s (a mixture: the expert gather does not halve); reading 501 × 2^0.75 ≈ 842 tok/s (assumed exponent). Two 128 GB M5 Max Studios ($10,198): at the default limit the pool offers 226.9 GB, enough for 215.2, so it fits tight (95 %), and runs at 84 % with the override on both machines (255.8 GB); at 45.2 ms per token for one Studio, ≈ 20 tok/s on a layer split and ≈ 30 in tensor parallel (221.7 GB, tight at 98 %). The sheet then names the comparison: one 256 GB M5 Ultra Studio ($9,499) runs the same file as asked (85 %) at ≈ 31 tok/s (estimated), $699 less than the pair. And the everyday model on two 24 GB M5 Pro minis ($3,398): it runs (23.0 of 34.1 GB) at ≈ 12 tok/s on a layer split, slower than one 48 GB mini ($2,299) writes it at 12.6; tensor parallel would put the pair at ≈ 23. Link only when nothing single fits.

10. Changelog

• 0.17.3 — 2026-10-19. Sparse attention on MLX (section 6.1). Qwen 3.8 Flash Next’s attention layers read 2,048 cached tokens per word, picked by an index, not the whole context; the tool read all of it, so its MLX estimate fell far too fast with context: on the oQ4e build 53 tok/s at 32K and 39 at 128K on an M5 Max, where oMLX measures 68.5 and 64. On MLX the cache read and the attention cost now stop at the 2,048-token budget, and an index cost per 32K of the whole context is fitted on three new validation.json rows (the medians of 14, 15 and 4 plain oMLX runs at 32K, 64K and 128K). It comes out at zero: with the 6.67B-parameter count of 0.17.1 the read over the budget and the fixed cost already account for those runs. The M5 Max estimate now holds at 59.4 tok/s from 4K to 128K, 7 to 14 % under the runs at every length; the mlx-community 4-bit file reads 58 tok/s at 4K, 32K, 128K and 256K on an M5 Max (52 and 38 at 32K and 128K before) and 48 on the M4 Max (35 and 19). The rule is keyed on the model type (constants.ts › SPARSE_ATTENTION, mirrored in speed_model.py) and needs no export; llama.cpp files keep the whole-context charge. The hypothesis it replaces, a context term scaled by the share of attention layers, is written up in section 6.1 and rejected: Flash Next’s share equals Qwen3-Coder-Next’s and the 27B’s, which already fit their long-context ladders, and on the M1 to M4 rows the scaled term makes the long-context median worse. 0.17.2 and 0.17.3 were built alongside 0.16.1 to 0.17.1 and are released with them; the factors are refitted on all 236 rows together. Against 0.17.1 that moves the pre-M5 MLX context cost from 6.4 to 6.3 ms per 32K and the MLX factor of the bigger mixtures from 0.77 to 0.78, and the Flash Next rows of the 128 GB comparison by 0 to 2 %; the held-out median stays 1.13 on the same 64 rows, and the training split grows to 137 rows. Known gaps: section 8, item 30.

• 0.17.2 — 2026-10-19. Qwen 3.8 Flash Next’s MLX files are counted the way Mac engines run them (section 5). oMLX, MTPLX and mlx-serve leave the model’s 51B-parameter n-gram table on the SSD, so the fit table, the buying map and “What can it run?” now count 71.7 % of an MLX file as weights and show the rest beside the memory bar as the SSD part. With MLX on, the 111.5 GB 4-bit file needs 86.7 GB at 32K and runs on a 128 GB Mac at the default limit, where it showed a compromise before (about 118 GB, over the 115.4 GB macOS gives the GPU). The 128 GB comparison already counted it that way and now uses the same rule, so the two agree by construction. The rule is keyed on the architecture and needs no export; GGUF files, an engine that loads the table into memory and machines with no MLX engine (the DGX Spark) count the whole file as before. These files keep the normal speed estimate, which the oMLX runs with the table on the SSD were fitted on. Four oMLX peak readings join validation.json as memory rows (the estimate sits 1.3 to 3.9 GB above them at 4K, section 8, item 14), and a cell says in a flag that the table stays on the SSD.

• 0.17.1 — 2026-10-19. Three active-parameter counts follow what a token reads (section 8, item 15). The tensor tables of the models’ own files give Qwen3-Coder-Next 3.56B per generated token (the catalog had the 3B of its name), Qwen 3.8 Flash Next 6.67B (6B: Qwen’s figure leaves out the output layer) and GLM-5.2 40.3B (54.0B from the config, which counted the multi-token-prediction layer, the input embedding and the shared experts twice). Reading speed follows the count: Qwen3-Coder-Next reads prompts 16 % slower and GLM-5.2 34 % faster, both closer to the prompts the oMLX community measured (Coder-Next at 4K on eight chips: median 20 % too fast → 7 %; GLM-5.2 on the M3 Ultra: 49 % → 32 % too slow); Flash Next’s reading class is fitted on its own sweep, so it moves 1 %. The refit with the new counts moves Flash Next’s llama.cpp fixed cost from 11.7 to 11.1 ms, the MLX factor of the bigger mixtures from 0.68 to 0.77, the MLX context cost from 6.9 to 6.4 ms per 32K before the M5 and from 0.9 to 0.7 on it, and the cost per expert layer on the M4 from 0.18 to 0.16 ms. Writing speed at 4K across the Macs: Qwen3-Coder-Next −10 to +2 % on MLX and −4 to −9 % on GGUF, Flash Next −8 to −9 % on MLX and 0 to −6 % on GGUF, GLM-5.2 +19 to +31 %; the other big mixtures on MLX (Qwen3.5-397B, Kimi K2.6, Nemotron 3 Super, DeepSeek V4 Flash, GLM-5.3 Flash) write 2 to 5 % slower, and the small mixtures on M4 chips 3 to 6 % faster. Against the measured runs writing gets a little worse on these three models, because the fixed costs had been fitted around the old counts: Flash Next’s median miss over its 11 rows goes from 1.07 to 1.16, Qwen3-Coder-Next’s over 23 rows from 1.05 to 1.07, and GLM-5.2’s one run from 2 % under to 18 % over. The held-out median goes from 1.124 to 1.126 and the training median from 1.06 to 1.07, inside the release rule. In the 128 GB comparison Flash Next on the M2 Ultra now reads 18 % under its run, so the run is shown with a reason; the M5 Max reads 14 % under (was 6 %), the M4 Max 4 % under (was 5 % over). Flash Next’s rows in validation.json are now checked against its catalog record like the others’.

• 0.17.0 — 2026-10-19. Small mixtures of experts on MLX are charged by their expert layers (section 6.2). For a mixture under 6B active on MLX the fixed cost per token is now the model’s expert layers times a cost per layer for the chip generation (M1 0.35 ms, M2 0.33, M3 0.21, M4 0.18, M5 0.15; the M6 borrows the M5’s), fitted on 50 rows, in place of the chip’s llama.cpp fixed cost times a group factor. The old rule scaled the chip’s fixed cost from the llama.cpp table, half a millisecond on the M5 Max, where gpt-oss-120b costs about six milliseconds a token beyond its read on MLX. What moves, against the measured runs: gpt-oss-120b on the M5 Max 121 → 89 tok/s (measured 83 to 90), gpt-oss-20b on the M5 Max 146 → 125 (124), on the 60-core M3 Ultra 82 → 109 (116) and on the M4 Max 84 → 105 (110); Qwen3-Coder-Next on the M3 Ultra 61 → 73 (79); Qwen3.6-35B-A3B on the M5 Max 94 → 103 (146, still the largest miss). It moves the wrong way for gpt-oss-20b on the 10-core M5, 43 → 54 against a measured 38, and on the M4, 34 → 38 against 34 (section 8, item 27). Across every Mac at 4K the five small mixtures in the catalog move by −26 % to +40 % (median: gpt-oss-20b +7 %, gpt-oss-120b +2 %, Qwen3-Coder-Next +16 %, Qwen3.6-35B-A3B +26 %, Gemma 4 26B-A4B +7 %); dense models, the bigger mixtures, Flash Next and every GGUF number do not move. The held-out median goes from 1.15 to 1.12 on the same 64 rows, and from 1.12 to 1.07 on the small mixtures in the training rows. A term for the GPU bin was fitted and left out: it helped the training rows and hurt the held-out ones. In the 128 GB comparison gpt-oss-120b now lands within 8 % on the M5 Max, M4 Max and M2 Ultra; the M5 Max’s and M2 Ultra’s notes on plain mixtures are gone, and the M5 Ultra’s says why its gpt-oss-120b run reads high. validation.json rows carry moe_layers (checked against the catalog) and, where the source states it, gpu_cores; the sheet’s “how this number is made” names the expert layers and the cost per layer.

• 0.16.1 — 2026-10-19. The speed model is refitted on 81 more oMLX community runs, one row per model, chip and GPU bin (gpt-oss-20b on 28 chips and GPU bins from the M1 to the M6, Gemma 4 26B-A4B on 23, Qwen3.6-35B-A3B on 15, Qwen3-Coder-Next on 11, gpt-oss-120b on 3 including its MXFP4-Q8 file on the M5 Ultra, and Flash Next on the 80-core M5 Ultra), each the median of the plain sessions within 10 % of that group’s median, with every run id in validation.json. Qwen3.6-35B-A3B and Gemma 4 are held out as whole families. No formula changes; the MLX factors for small mixtures move from 1.13 to 1.12 (M4 and newer, now 20 rows) and from 1.2 to 1.18 (M1 to M3, now 30 rows), so no speed moves by more than 2 %. The held-out median is 1.15 on 64 rows; the 23 rows held out in 0.16 still score 1.11. Two findings are written up (section 8, items 27 and 28): on MLX a small mixture’s cost per token follows its expert layers and the chip generation, not the chip’s llama.cpp fixed cost; and a smaller GPU bin writes mixtures 0 to 10 % slower, not the quarter 0.16 reported for the 48-core M1 Ultra, whose gpt-oss-120b runs date from April and an older oMLX. The 128 GB comparison’s notes for the M1 Ultra say so.

• 0.16.0 — 2026-10-19. “Every way to 128 GB”, a seventh tile (section 6.6): one model at one build on every route to about 128 GB, from a gaming PC with system memory to an RTX PRO 6000, with the price, memory speed, usable memory, fit, writing speed, a 1,000-word answer’s time and the measured reading time for 50 pages, from a dated, sourced data file (routes-128gb.json), and a calibration table of every published plain run against the estimate. The speed model is refitted on 28 new rows in validation.json. The MLX factor for small mixtures of experts on the M4 and newer, assumed at 0.5 from one 256-token run, is fitted on seven rows at 1.13: gpt-oss-120b, Qwen3-Coder-Next, Qwen 3.6 35B-A3B and Gemma 4 26B-A4B write 9 to 41 % slower on the M4 and M5 Macs under MLX (most by 15 to 30 %), which is what the oMLX runs measure (gpt-oss-120b on a 128 GB M5 Max goes from 102 to 88 tok/s at 32K; measured 83 to 90 at 4K). Qwen 3.8 Flash Next becomes its own architecture class (11.7 ms, three llama.cpp rows), so it writes 13 to 55 % slower on the Macs than before (30 to 55 % on the Max and Ultra chips), within 15 % of the oMLX runs on the M5 Max, M4 Max and M2 Ultra; the other mixtures’ MLX factor moves from 0.77 to 0.68 with Flash Next’s rows in it, and the MLX context costs from 6.4 to 6.9 ms (M1 to M4) and 0.8 to 0.9 ms (M5) per 32K. The held-out median improves from 1.14 to 1.11. Two gaps are written up instead of fitted: plain mixtures on the M5 generation (section 8, item 27) and GPU cores on mixtures (item 28).

• 0.15.0 — 2026-10-15. Reading speed refitted on measured runs. The M5 Ultra reviews and the video graphics that quoted them showed the tool’s reading speeds at about half of what owners measure on MLX: Qwen 3.8 27B in the 4-bit MLX file read at 933 tok/s on the 80-core M5 Ultra in the tool against 1,701 in MacStories’ review, 466 on the M5 Max against 912 to 1,022 in oMLX runs at a 4K prompt, 233 on the 20-core M5 Pro against 508 to 518, and 240 on the M3 Ultra against 414. The cause was the MLX factor: 0.61 on the M5 and 0.65 before it, fitted on MLX runs at a 512-token prompt, six of them from one fanless MacBook Air in April 2026. Refitted on runs of 4K tokens and more (the oMLX community benchmark database, MacStories, mlx-serve), MLX reads at 0.80 of llama.cpp’s plain-file constant on the M1 and M2, 1.27 on the M3 and M4 and 1.31 on the M5 and newer, in three groups where there were two (section 6.4). The halving depth of the context term (D0) is now fitted per runtime on measured ladders: 81,781 tokens on llama.cpp (Tom’s Hardware’s M5 Ultra, M4 Max and DGX Spark ladders) and 95,613 on MLX (11 oMLX ladders), where one Flash Next sweep had set 19,257 for everything; Flash Next keeps that value on llama.cpp. What moves, for Qwen 3.8 27B on MLX: the short-prompt rate about doubles on the M3 to M6 chips (1.95 to 2.15 times: M5 Ultra 80-core 933 → 2,003, M5 Max 466 → 1,002, M5 Pro 233 → 501, M3 Ultra 80-core 240 → 468, M6 140 → 300) and rises 1.2 times on the M1 and M2; the 8K wait falls 2.3 to 2.5 times on the M3 and newer (M5 Max 21.3 → 8.5 seconds, measured 9.9) and about 1.4 times on the M1 and M2. On llama.cpp the short-prompt rates do not move and every long prompt waits less (a 32K prompt on the M5 Max 1.5 → 1 minute, a 128K one 14.3 → 5.8 minutes). The latent-attention class (GLM-5.x, Kimi, DeepSeek) gets its first measured reading factor, 0.48 where it borrowed the mixture factor 0.68, from MacStories’ GLM-5.3 Flash runs, so those models read 0.7 times as fast as the new MLX factor alone would make them: GLM-5.3 Flash on MLX on the 80-core M5 Ultra reads 1,484 tok/s at a short prompt (979 before; MacStories measured 1,107 over 13,639 tokens), and on a DGX Spark it reads 501 instead of 709 (worked example H). Writing speed, memory, verdicts and prices do not move, and the writing-speed gate is unchanged (holdout median ×1.14, passed); the first M5 Ultra and M6 runs write within 6 % of the assumed profiles on MLX, and on llama.cpp except at 64K (12 %; section 6.1). New data: benchmarks.json gains prefill_ladders (Tom’s Hardware’s three time-to-first-token ladders read from the chart image, MacStories’ two GLM-5.3 Flash ladders, one oMLX benchmark submission, and 44 oMLX community ladders whose points name every submission), prefill rows carry prompt_tokens and app, and report_only rows quote setups the fit cannot use (oMLX’s INT8-activation prefill, a spoken M5 Pro figure, Alex Ziskind’s M6 and M4 llama.cpp runs). factors.json › prefill gains context.mlx, context.by_model, the per-ladder fits in ladders, and the excluded MLX rows. Section 8 gains items 22 to 26 on what the fit still gets wrong.

• 0.14.0 — 2026-10-15. The macOS memory limit follows macOS 26. The tool assumed macOS lets the GPU use two thirds of memory under 36 GB and three quarters from 36 GB, the values logged on macOS 14 and 15. On macOS 26 Apple gives a bigger share that grows with the machine: 67 % on 8 GB (unchanged), 74 % on 16 to 24 GB, 78 % on 32 to 48 GB, 81 % on 64 and 96 GB, 84 % on 128 and 192 GB, 87 % on 256 GB, and all but 48 GB on 512 GB. The values come from more than 50 public logs across every size and a reading on the maintainers’ 128 GB M5 Max (macOS 26.6.2: 115,448,725,504 bytes, exactly 84 %); section 4 has the details and section 8 (item 21) the thin spots. The old rule was already low at 256 and 512 GB even for recent versions of macOS 15, which hold back at most 48 GB there. Memory available for AI on an empty Mac rises by 0.4 GB on 18 GB, 1.9 on 24, 3.9 on 32, 1.2 on 36, 1.5 on 48, 4.1 on 64, 6.2 on 96, 12.4 on 128, 18.6 on 192, 33 on 256 and 86 on 512; 8 and 16 GB do not move, because the 6 GB kept for macOS runs out first. Across 64 models, every Mac size, 8K, 32K and 128K, apps off and on, and both runtimes, 6,439 of 85,248 verdicts change and none gets worse: 795 rings become gold spheres and 3,040 no longer need the memory-limit override. Examples: Qwen 3.8 27B at Q4 and 32K now runs on a 32 GB Mac (77 %, was tight at 90 %); GLM-5.3 Flash at Q4 and 128K runs on a 256 GB Studio (86 %, was tight at 99 %); GLM-5.2 at Q4 fits a 512 GB Studio without the override (97 %); two 128 GB M5 Max Studios fit GLM-5.3 Flash on a layer split without it (95 %). The compromise search is unchanged, so where a smaller build now fits without the override it wins as before: on a 128 GB M5 Max, GLM-5.3 Flash’s fix becomes the 2-bit build instead of the override plus the 3-bit build. The switch and Advanced now read “67 % to 91 % by size”; on macOS 15, set 67 % or 75 % in Advanced. Against the video’s charts, GLM-5.3 Flash on 256 GB now matches the 128K chart’s sphere, and runs where the realistic chart drew a ring; the other correction notes carry the new numbers. Worked examples A, B, C, F, G and H are updated. Speeds, prices and every other formula and constant are unchanged.

• 0.13.4 — 2026-10-14. The sixth tile, “How fast is every Mac?”: one model on every Apple chip from the M1 to the M6 as one grid, generations across and tiers down, the way the speed charts in the videos read (section 1). It shows writing speed, or the wait before the first word for a prompt as long as the chosen context; any catalog model can be picked, at Q2 to Q8 and at each context chip the model allows, with the runtime, the memory limit and room for your apps taken from Settings as everywhere. Each chip is shown at the biggest memory it was sold with or at the smallest that runs the model as asked (falling back to the smallest that runs with the limit lifted); tapping a chip lists every memory size with its chip version, both speeds and the list price, and a size opens the usual sheet. New chips (an M6 Pro, an M7) appear from the catalog without a code change. The share link carries the view (vchips*<model>*<build>*<context>, then r for reading and n for the smallest memory); the share image and presentation mode show the table for it. No formula or constant changed.

• 0.13.3 — 2026-10-05. DGX Spark prices and a second Spark size. NVIDIA’s own store now lists the 128 GB DGX Spark at $6,950 (it launched at $3,999 and was $4,699 from February), so the Spark’s price in the tool rises by $2,251 and two linked Sparks cost $13,900. At that price it costs more than a 128 GB Mac Studio M5 Max ($5,099) or MacBook Pro M5 Max ($6,699), so it now sorts after them on the buying map and falls outside a $6,000 budget. NVIDIA also announced a 64 GB DGX Spark on 2026-10-02: the same GB10 chip, 256-bit memory and 273 GB/s, sold only by Acer, ASUS, Dell, Gigabyte, HP and MSI from 2026-10-23, starting at $4,999. It joins the Spark as a second memory size, with 60.7 GB available for AI after the 8 GB reserve and the same writing and reading speeds as the 128 GB model. Like any size not on sale yet it has no price in the tool, so the buying map lists it when it qualifies but never calls it the cheapest; it gets a price once partners sell it. The new size appears by itself on the first screen and on older links that include the Spark, as any size added to the catalog does. Sources are in the Spark’s row: NVIDIA’s store and specifications, NVIDIA’s announcement, and XDA (2026-10-02). No formula or constant changed, and Mac prices were not re-checked.

• 0.13.2 — 2026-09-24. Reading speed and price now describe the same machine. A reader pointed out that the 48 GB Mac mini M5 Pro showed the $2,299 price of the 16-core GPU with the reading speed of the 20-core one, a $200 upgrade. Where a row covers two GPU bins, the reading estimate now uses the bin the list price buys at that size (section 6.4): the smaller one, unless the size only comes with the larger. The sheet names the bin beside the price and the reading speed and adds a line for the larger bin with its rate, its 32K wait and Apple’s price for it; the Reading tab’s header says the same. Reading numbers fall by the core ratio on those rows: a fifth on the M5 Pro mini, the 24 and 48 GB MacBook Pro M5 Pro, the M5 Ultra Studio at 96 and 256 GB, the 16 GB MacBook Air and the 16 and 24 GB iMac, and an eighth to a quarter on the older two-bin chips. New catalog fields gpu_cores_by_gb and gpu_upgrade_usd, checked by validate_machines.py, with the upgrade prices read from Apple’s configurator on 2026-09-24. Writing speed, memory and prices do not move.

• 0.13.1 — 2026-09-15. The code is open source, as promised in the launch video at 100,000 views: the whole tool is public under the MIT license beside the open-data repository, and runs on its own from a snapshot of the data. The site keeps its own copy of one adapter file (the brand, the author line, the email card, the feedback inbox), and the public copy ships a neutral one. The footer and the About page link both repositories. No formula or constant changed.

• 0.13.0 — 2026-09-08. Linked machines. A machine group can be pooled ×2 to ×4 (pick it again in “Add machine”, or the chips in Settings) with a layer-split or tensor-parallel switch; the row reads “2 × … · 128 GB each · 256 GB pooled” with the summed price on every view, and the share link carries it (x2, x4t on the group item). Memory pools per machine (section 4): each keeps its own reserve and GPU limit, the work-apps reserve is taken once, 2 GB per machine goes to the link, shards pack 5 % (layer) or 8 % (tensor) looser and the cache is duplicated under tensor parallel on latent-attention models (section 5). Writing speed follows the two measured regimes (section 6.1): one machine’s time for the whole model plus a hop per extra machine on a layer split (4.0 ms Macs, 3.9 ms Sparks, fitted on the published series), or one machine’s time ÷ N plus 4.5 ms (dense) or 10.9 ms (mixtures) × log2 N in tensor parallel; reading speed stays one machine’s on a layer split and scales as N^0.75, assumed, in tensor parallel (section 6.4). The sheet names the cheapest single machine that runs the same model as asked, so two boxes never quietly beat one that fits. New data: benchmarks.json › cluster_series (twelve published series), factors.json › cluster from calibrate.py, speed_model.py mirrors and a parity fixture. Sections 7, 8 (items 17 to 20) and 9 (example H). Single-machine numbers do not move.

• 0.12.1 — 2026-09-07. Two labels, after viewer questions on the launch video. The Reading tab now opens on an 8K prompt instead of the column’s whole context, its last chip reads “the whole context” and says it is a prompt pasted in one go, and the tab and the sheet say that in a chat each turn waits only for its new text, since what the model has already read stays in memory (section 6.4): a 256K one-shot read on a 27B dense model is over an hour on an M5 Max, and that is the physics, not a per-message cost. The sheet’s memory section now says the context cache is counted at FP16, the runtimes’ default, and what an 8-bit or 4-bit cache would make it (section 5); the setting itself stays under Advanced, off the table and the charts. The About page’s stale “not yet” answer on prefill is gone. No formula or constant changed.

• 0.12.0 — 2026-09-06. Reading speed. Every cell now carries a prefill estimate (section 6.4): the wait before the first word for 8K, 32K and 128K prompts and for the column’s own context, from a per-generation per-core constant fitted on the community llama.cpp table (the file’s bits do not matter, the GPU cores do, the M5 generation reads 3.5× the M4 per core), a K-quant factor, an architecture-class factor, an MLX factor and one context-depth term fitted on the published Flash Next sweep. “How fast?” gains a Reading tab with a prompt-size switch; the sheet gains a “Reading speed” section. Twenty community prefill rows (mac-llm-bench, MIT) and the depth sweep join benchmarks.json with their sources; speed_model.py mirrors the formula and a parity fixture pins both sides. Writing speed and memory are unchanged.

• 0.11.0 — 2026-09-06. The engram-on-SSD build of Qwen 3.8 Flash Next is a build you can pick, not only a fallback: it is in the column’s build list under the files, an “SSD build” segment on the buying map, and the machine view’s best build when no file fits as asked. The data now carries the three AtomicChat builds with what stays in memory and what streams from the SSD (45.8 + 39.1, 54.5 + 38.4, 56.1 + 54.4 GB), replacing one entry whose sizes did not reconcile, and the measurement’s chip (M5 Max). The memory bar shows the SSD part beside it; the sheet, the flags and the fix text say “engram on SSD” instead of “SSD-paged, resident”. Marks with no speed figure qualify on the buying map when any speed will do and never meet a minimum. A share link writes such a column as s<n>; older links open as before. The memory and speed formulas are unchanged; the special-build record gained two fields.

• 0.10.5 — 2026-09-06. A viewer compared a 24 GB and a 48 GB MacBook Pro M5 Pro in “How fast?” at Q8 · 256K and saw the smaller machine “faster”: both rows were compromises (the 24 GB fell back to a 4-bit file at 8K, the 48 GB kept the 8-bit file at 32K), and a 4-bit file reads half as much per token. The verdicts and speeds were right; the presentation hid the swap behind a small “at the fix”. A compromise row in the speed view is now drawn as a dashed outline instead of a solid bar, its number says “at the fix”, its label names the configuration it runs on one line and the one you asked for (“did not fit”) on the next, and the footnote says why a smaller machine can show a higher number. No formula or constant changed.

• 0.10.4 — 2026-09-06. The page names its author under the title, with a link to the YouTube channel. The email card no longer opens on the first click or after 45 seconds: it waits for three minutes of active time on the page and at least two real interactions, stays away for a week when dismissed (a month before), and never returns once you have subscribed. No formula or constant changed.

• 0.10.3 — 2026-09-06. The “I own” picker under each machine in Settings is gone, with the “yours” tag, the dashed ring and the keep-your-machine answer it drove: the same rules now apply to every machine on the buying map. In its place the map shows every current machine, every older catalog machine you put in the table (so an M1 Max you already have still appears, without a price) and your own machines. Older links that carried an owned machine still open; that section is ignored. The model picker’s middle tier is labelled “Studio-class (64–256 GB)” instead of “Big Mac / Studio”. No formula or constant changed.

• 0.10.2 — 2026-09-05. The buying map’s speed axis is numbered at the feels-like band edges (0, 5, 12, 30, 60 and the end of the axis) instead of round ticks, so every band’s border carries its number; the “cheapest that qualifies” and “yours” tags are measured with their labels, so a tag can no longer run into the memory-size column. No formula or constant changed.

• 0.10.1 — 2026-09-05. The tool went public: the tool and this page are indexed, in the site map and in the site’s navigation (a phone menu was added for it). No formula or constant changed.

• 0.10 — 2026-09-05. The MLX long-context cost is now per chip generation (sections 6.1 and 6.2): 6.4 ms per token per 32K on the M1 to M4 chips as before, 0.8 ms on the M5 generation, fitted on the oMLX M5 Max ladders. The old single value, fitted only on M3 Ultra and M1 Ultra sweeps, charged the M5 chips a penalty they do not show: the tool under-predicted the M5 Max by 17 to 21 % at 32K to 64K on MLX, and the M5 Ultra by about 45 % at 256K, so the Ultra-to-Max ratio at long context read 1.4× where the bandwidth ratio says 1.8×. The two oMLX M5 Max submissions that run past 8K moved to the training split to fit the new term (holdout 13 rows, median ×1.14, gate passed; train 79 rows, median ×1.05). Section 8 gains item 16 on how thin that fit is. Numbers at 32K and below, and every GGUF number, do not move.

• 0.9.6 — 2026-09-05. The buying map after the founder’s first look: the “doesn’t fit” gutter is gone (a dash just left of the zero line, never on the speed axis); machines at the same speed get a label each, one under another (a joined label only past three or where nothing fits beside them); a legend above the map explains the five marks; the answer card and the list under the map are retired, the map itself names the cheapest qualifying configuration and the machine you own, and a single line says when nothing qualifies. The cell sheet is in sections: the list price first, the memory bar without the sentence that repeated it, the speed as a number with the feels-like phrase and the acceleration ceiling, the estimate’s provenance on hover and on demand, the sources on one line. No formula or constant changed.

• 0.9.5 — 2026-09-05. “Which machine?” is now a buying map (section 1): every current machine configuration placed by memory size and estimated writing speed for one model at one build and context, a Q4 / Q8 / own-build switch on a shared axis, a minimum-speed line, and three new settings (a budget, laptop or desktop, the machine you own) that turn the marks into a recommendation with the rules written down: fits as asked, speed at or above the minimum, within the budget, matching the preference; a compromise is a ring at its fix’s speed and never qualifies; a machine that does not fit is never placed at a speed; an unpriced machine is never called cheapest; nothing qualifies is said plainly with the nearest misses. Worked example G. The share link gains the budget, the preference and the owned machine; links made before the map open it at its defaults. No formula or constant changed.

• 0.9.4 — 2026-09-05. “What can it run?” tidied after a second look: the data columns are spaced apart and a row says one thing per column (a dash reads “Doesn’t fit”, the reason moves to the tooltip and the sheet); the whole row opens the sheet; size bands are headed in full (“40 to 120 GB at Q4 · 6 models”); the expanded table has a proper header (“Build” over the rows, “Context window” over the columns) and vertically centred row labels. No formula or constant changed.

• 0.9.3 — 2026-09-05. “What can it run?” reworked after the founder’s first look: models are grouped by the size of their Q4 file instead of the catalog’s laptop / studio / frontier tiers; every row carries the parameter count and the memory the configuration in the row needs; the expanded view is a full-width table per model (rows = builds, columns = context windows, speed and memory under each mark) under a tinted header band that repeats the model’s summary, with a legend above the list. No formula or constant changed.

• 0.9.2 — 2026-09-05. The fifth tile, “What can it run?”: one machine at one memory size against the whole catalog, best build first, expandable into every build and context (section 1). Any catalog machine can be picked, including older ones and the machines typed into Advanced; the assumptions in Settings apply as they do everywhere; a cell opens the same sheet, where only the global fixes (closing your apps, lifting the limit) can be applied because a build or context is a cell of the grid, and “Which machine do I need?” adds the model to the table at that build and context. The share image and presentation mode show the table for this view. A “Show benchmarks · coming soon” chip records interest through the notify card. No formula or constant changed.

• 0.9.1 — 2026-09-05. Corrections from a second independent audit of the 0.9 release. A hand-maintained build’s published speed is no longer used as the cell’s number on any machine (it had been shown on every Apple machine with a 64 GB option); it is quoted as a note with its source, and the sheet’s file link now follows the build that fits. Active parameters: MiniMax M3 23B (was 49B, the config hides the expert width), Kimi K3 104B (was 165B, the experts work on a 3,584-wide latent), Qwen3-Coder-Next 3B (was 4.0B), Gemma 4 E4B 4.5B effective (was 8B: the per-layer embedding tables are lookups), each from the model card; the parser now reads a latent expert width and Gemma’s embedding tables itself. Validation data: four rows transcribed from the wrong context of one source corrected, memory readings now carry their published unit and metric with oMLX’s binary gigabytes converted (the 27B MLX residual at 64K is 1.7 GB, not 3.7), the file sizes of the mixture-of-experts memory rows added so the memory report covers every row, three readings taken at a configured rather than filled context moved to report-only, and a test that refuses a validation row whose active count or cache bytes differ from the catalog record. The re-fit moved no term: held-out median ×1.14, gate passed. Section 8: item 14 corrected, item 15 added.

• 0.9 — 2026-09-05. The speed model is now a fixed-cost model: milliseconds per token = a read of the bytes per token at the chip’s effective bandwidth + a fixed cost (the chip’s, fitted from the F16/Q8_0/Q4_0 rows of the community table, plus an architecture-class cost fitted on measured runs) + a context term; tok/s is its inverse (section 6.1). MLX now scales the fixed cost by model group instead of multiplying the speed, GGUF K-quant files on Apple silicon carry a 1.25 read factor, and the MLX long-context penalty is a fitted term instead of a 24K–36K fade (section 6.2). Fitted on 73 measured runs and judged on 19 held-out ones: median error ×1.14 against ×1.21 for the multiplier model, with no row getting worse. What moves: mixtures of experts with a small active set lose the 1.5–3× the old model gave them (GLM-5.3 Flash on the M5 Ultra ≈ 35 tok/s, was 59; its 3-bit build on the Spark ≈ 15, was 25), 8-bit dense files on the Ultras gain, and the Mac-vs-Spark contrast on GLM-5.3 Flash shrinks from about 4× to about 2.3×. Memory margins became explicit policy (sections 4 and 5): buffers are 1.5 GB plus 1 % of the weights instead of 5 % capped at 16 GB (a 467 GB file is charged 6.2 GB, not 16), MLX files carry 0.15 GB of prompt scratch per 1K tokens beyond 8K as their own bar segment, and with the GPU limit lifted macOS keeps the larger of 6 GB and 5.5 % of the memory (30 GB on a 512 GB Studio). New subsection in section 1 on the five kinds of number. Worked examples rewritten: A is tight at 90 % (the buffers grew), B is tight at 99 % without the override, C at 93 %, E’s nearest miss is 724 against 520 GB. The aired-chart cross-check disagrees in 17 of 108 cells (the realistic chart’s 48 GB Qwen cell is tight at 92 %). Data: validation.json (120 measured speed and memory rows with sources, grades and the train/holdout split) joins the mirrored files; benchmarks.json gains the F16 and Q8_0 rows per chip (role floor-fit, outside the retired multiplier’s medians).

• 0.8 — 2026-09-05. Verified defects from two independent audits against measured runs. Memory sizes are now converted from the binary gigabytes Apple and NVIDIA quote to the decimal gigabytes file sizes use before any reserve is subtracted (32 GB = 34.4 GB, 512 GB = 549.8 GB); the macOS GPU limit is ⅔ under 36 GB and ¾ from 36 GB, as Metal’s logged working set shows, not ¾ from 64 GB (36 and 48 GB Macs gain about a fifth of their usable memory). The context cache defaults to FP16, which is what every runtime stores unless told otherwise (it was 8-bit); Gemma 4’s shared key/value projection no longer halves the stored cache (26B-A4B ≈ 1.4 GB at 8-bit, 2.9 GB at FP16, was 0.7); a hand-maintained build’s measured speed is shown only on the machine class it was measured on. DGX Spark efficiency is 0.75 from five llama.cpp maintainer rows (build b7941); it was 0.40 from a single row that was recorded as a 40 GB Q4 llama.cpp run but was in fact LMSYS’s Llama 3.1 70B at FP8 under SGLang (kept in the data for provenance, excluded from calibration) — every Spark speed rises about 1.9×. Mistral Small 4’s active parameters are 6.5B per its model card (the config-derived estimate was 8.0B). Worked examples rewritten: the 256 GB Studio fits GLM-5.2’s 2-bit build with the override (D is a ring), the 64 GB MacBook Pro runs Flash Next’s SSD build without the override (F), the Spark fits GLM-5.3 Flash’s 3-bit build (B). The aired-chart cross-check now disagrees in 16 of 108 cells, none from a formula error. The MLX fade end is single-sourced at 36K (the data file said 30K).

• 0.7 — 2026-09-04. The memory chart became “How much memory?”: absolute bars of what a configuration needs (weights + context cache + buffers), machine-independent, so the same model at two builds or two context lengths can be compared; one machine’s limit is a dashed reference line. No formula or constant changed.

• 0.6 — 2026-09-03. The simple UI: all configuration moved into a settings panel (machines with tickable memory sizes, models as one-build-one-context columns, drag to reorder, the two switches, Advanced folded in); four chart tiles (fit table, speed, which machine, memory); the memory breakdown became a full chart; a Feedback button and a closable notify card replaced the forms under the table. Section 1’s reading guide updated. No formula or constant changed.

• 0.5 — 2026-09-03. The About page (/tools/llm-sizer/about) renders this document with a FAQ and the sources behind every machine, benchmark row and factor; every number in a cell’s sheet now links to its source (section 1). Correction recipe by file (section 11). Colours inside the tool darkened for contrast (the video’s palette stays on the filming canvas). Cross-check against the charts that aired in the Apple-Macs video (2026-08-29), reproduced through the tool’s own table path and kept as a test: the tool disagrees in 14 of 108 cells, none from a formula error. The reasons: the chart drew “does not fit at Q4” as a dash where the tool shows the cheapest compromise as a ring (a 2-bit or 3-bit build, a shorter context, the memory-limit override, or closing your apps in the realistic chart); the chart assumed the three-quarters memory limit on a 32 GB Mac where macOS allows two thirds (Qwen 3.8 27B at Q4 + 128K needs 22.2 GB against 21.4); Qwen 3.8 Flash Next’s Q4 file is 119 GB (the n-gram table is inside it), not the ≈ 85 GB the chart assumed; GLM-5.3 Flash at Q4 needs the override on a 256 GB Studio once buffers count; GLM-5.2’s 2-bit build does not fit 256 GB with its cache and buffers, and on 512 GB it needs the override and lands at 97 % (tight, a ring); and the speed table’s flat 0.48 efficiency is now per chip (section 6.1). No formula or constant changed.

• 0.4 — 2026-09-03. The app. New “How to read the table” subsection (section 1): the three marks, the two switches, tight cells as rings, speed shown for the configuration that fits, links and custom entries. Fix descriptions now separate the tight note with a semicolon instead of a dash; the alternatives list never contains a superset of an earlier alternative. No formula or constant changed.

• 0.3 — 2026-09-02. The engine. Verdicts now have four states (runs / tight / compromise / doesn’t fit) with the fix search’s costs written down; the 2-bit quality floor (by label) and hand-maintained special builds added; MLX factor fades 24K → 36K instead of dropping at 30K; efficiency computed on active weights for mixture-of-experts benchmark rows (Ultra tier 0.44, M5 Ultra assumption 0.55); chip names matched without their GPU bin; ROCm machines get no speed number. Worked examples rewritten from the live data: Qwen Q4 is 16.8 GB (LM Studio’s file), GLM-5.3 Flash ≈ 59 tok/s at 8K on the M5 Ultra, its Q4 does not fit the DGX Spark (the 2-bit build runs ≈ 17 tok/s), GLM-5.2 on 256 GB is a dash once buffers are counted, Kimi K3 and Flash Next examples added.

• 0.2 — 2026-09-02. Data pipeline built. Section 3 now describes how models are discovered (five quantizer organisations, twelve-month window, featured list, name-based exclusions) and the publish-time safety net. Efficiency constants are now generated by calibrate.py from benchmarks.json into factors.json (tier medians 0.79 / 0.74 / 0.63 / 0.45 / 0.40; measured chips use their own value, M5 Max 0.82); M5 Ultra documented as an explicit 0.57 assumption. Acceleration ranges computed from paired rows. Gemma 4 cache corrected to its global-layer heads (≈ 0.7 GB at 128K, not 1.3). Machines catalog: one row per bandwidth bin (M5 Max 460 vs 614 GB/s; M6 mini 153 GB/s at 16 GB vs 170 at 24/32 GB). Spark speed in example B corrected from ~14 to ~11 tok/s.

• 0.1 — 2026-09-02. First draft, written from the Phase 0 research spike: efficiency by chip tier from the community llama.cpp table (M1–M5 Max), context-cache formulas by attention design read from model configs, MLX runtime factors from 2026 community reports, MTP/DFlash factors from MTPLX and mlx-dspark measurements.

11. How to correct us

Use the feedback form in the tool (it attaches your exact configuration) or open an issue on the open-data repository, ideally with: machine, model, quant, context, runtime, and the number you measured. Accepted measurements go into the calibration data with your handle as the source, and the relevant row above changes with a changelog entry.

File by file (the canonical copies of machines.json, benchmarks.json and models.overrides.json live in the tool’s repository and are mirrored to the open-data repository every night; propose a change as an issue or pull request there and it lands in the tool):

• A machine fact (memory options, bandwidth, price, status): edit the row in machines.json and add the page you are citing to its sources.
• A speed you measured: add a row to benchmarks.json with chip, bandwidth_gbs, memory_gb, model, quant, weights_gb (decimal GB; active_weights_gb too for a mixture-of-experts model), runtime (llama.cpp-gguf, mlx, or another runtime’s name — only llama.cpp rows calibrate the efficiency), accel (none, mtp, dflash, dflash2, dspark), tg (writing tokens per second), pp (prefill, optional), source (a public link), date, and where you have them build (the runtime build) and context_tokens (tokens already in context when you measured); accelerated rows also carry baseline_tg, the same setup without the acceleration; rows meant for the chip-profile fit (F16, Q8_0 and Q4_0 reads of one dense model at short context) carry role: "floor-fit". A measurement of a whole configuration (a model at a quant, runtime and context, with tokens per second or peak memory) goes into validation.json with a grade (A: the maintainers’ or a reproducible tool’s number; B: a community post with the setup stated; C: a headline number) and a split (train, holdout, or report for rows the fit must not learn from); a memory reading also records the number as published, its unit (binary or decimal gigabytes, or unknown) and what was measured (allocator peak, process footprint, a configured context), and the row’s active count and cache bytes must be the catalog’s. Do not edit factors.json: it is regenerated from the rows.
• A model fact the Hugging Face API cannot express (a published active-parameter count, a licence threshold, a special build with its resident_gb, ssd_gb, disk_gb, bits, url, note and the publisher’s measured_tok_s / measured_on / measured_chip): models.overrides.json. The export fails on a special build missing a label, a size or a known format.
• A formula or constant: open an issue. FORMULAS.md is a nightly copy of this document and is not edited in the data repository.