LLM Sizer — Methodology & Assumptions

Version 0.10.2 · 2026-09-05 · Status: public, early build. This is the public, plain-language explanation of every number LLM Sizer shows. It is the single source of truth: the tool’s About page renders it and the open-data repo’s FORMULAS.md is a nightly copy of it, and any change to a formula, constant, or data source in the tool must be reflected here first (that rule is part of the project’s definition of done). Comments and corrections: the feedback form in the tool, or an issue on the open-data repository. Changelog at the bottom.

1. What the tool answers — and what it doesn’t

LLM Sizer answers three questions for a machine you own or are considering: which open AI models fit in its memory, how fast they will write answers, and what you’d have to change to make a model fit. Every answer is a calculation from published data, calibrated against real measurements, and every estimate is labeled as one.

It does not run a benchmark on your machine, judge model quality (we show quantization-quality notes where quantizers publish them, nothing more), or cover image/video generation models (those are compute-bound and need a different method).

How to read the table

Machines are rows (one row per memory size, grouped by model line: “Mac mini · M6” covers the 16, 24 and 32 GB configurations), models are columns (one column per model at a chosen quantization and context length). Each cell is one of three marks:

• Gold sphere: runs. The whole configuration fits in the memory available for AI with at least 10 % headroom. The sheet behind the cell says how much of the available memory it uses.
• Gold ring: runs with a compromise. Something has to give, and the cell already picked the cheapest concession by our ranking (section 5): close the apps you reserved memory for, raise the macOS memory limit, shorten the context, or drop to a smaller build, in that order of preference. A cell that fits with under 10 % headroom is also a ring, labelled tight. The sheet lists the chosen fix, up to three alternatives (never “an earlier option plus something extra”), and applies any of them to the column with one tap.
• Grey bar: doesn’t fit. Nothing at or above the quality floor (2-bit builds by label, adjustable in Advanced) fits even with every concession. The sheet shows the nearest miss: the smallest allowed build and how much memory it would still need.

Everything that configures the chart lives in the Settings panel (a column on the right on wide screens, a button above the chart on narrow ones): the machines and which of their memory sizes to show, the models (each column is one model at one build and one context, set when you add it and editable in the list, which you can also reorder by dragging), and two switches. “I’ll also use it for work” reserves 16 GB for your own apps (adjustable under Advanced); it is off by default, so the default table is the best case with nothing else running. “macOS memory limit” shows the share of RAM macOS lets the GPU use (67 % under 36 GB, 75 % from 36 GB); the sheet’s fix and Advanced can lift it to the override (section 4). The context cache is counted at FP16 by default, as every runtime stores it unless told otherwise (section 5). Speed numbers are always for the configuration that fits: a ring’s speed is the speed of its fix, not of the build you asked for. Column headers carry the model’s total and active parameters (“180B · 6B active” for a mixture-of-experts model), the build (Q4 by default) and the context length (32K by default). Five tiles above the chart switch between the fit table, the speed bars for one machine, the buying view (“Which machine?”), which places every current machine configuration on a map of memory size (rows) against estimated writing speed (x, shaded into the feels-like bands of section 6.5) for one model at one build and context, with a Q4 / Q8 switch on a shared axis, a minimum speed (any, 5, 12, 30 or 60 tok/s), and, from Settings, a budget, a laptop-or-desktop preference and the machine you already own; a configuration qualifies when it fits as asked, has a speed estimate at or above the minimum, costs no more than the budget and matches the preference, and the map names the cheapest qualifying configuration with a current list price on its mark and marks the machine you own (the owned machine is exempt from the budget and the preference because it is not being bought); a compromise is a ring at its fix’s speed and never qualifies, a machine that does not fit sits in a gutter and is never placed at a speed, a fit with no speed estimate never qualifies, an unpriced machine (a custom one, or a size Apple has not priced yet) is listed but never called cheapest, and when nothing qualifies a line under the map says so and names the nearest misses instead of inventing a pick; every mark opens the sheet, which carries the machine’s list price, the memory bar and the speed with its estimate detail on demand, the memory chart, which shows what each model needs at its build and context, split into weights, context cache, buffers and, for MLX files at long context, prompt scratch, with one machine’s limit drawn as a reference line, and the machine view (“What can it run?”), which takes one machine at one memory size and lists every model in the catalog, grouped by the size of its Q4 file (under 40 GB, 40 to 120, 120 to 256, 256 to 512, over 512), with its parameter count, the highest-quality build that runs as asked at 32K, the memory that configuration needs on the machine, and the longest context that runs at Q4; when nothing runs as asked it names the cheapest compromise or the nearest miss. “Show every build and context” turns each model into a table whose rows are the quantization buckets and whose columns are the context windows, with the same three marks and, under each, the speed and the memory needed; a build the model has no file for is shown as a gap, never as a substitute file.

Every table state is a link (?s=), the share image is rendered from the same math on the server, and a custom model or machine you type in travels inside the link so the person you send it to sees exactly your table.

Every number in the sheet behind a cell links to its source: the weights to the file’s repository on Hugging Face, the machine to its row in the sources table (bandwidth, prices, spec pages), the context cache and the available memory to the sections of this page that compute them, and the speed to the chip profile it used (effective bandwidth and fixed cost per token), with a sentence saying whether that profile was fitted from this chip’s own rows, assumed, or taken from the chip’s tier.

What kind of number is this

Every figure the tool shows is one of five kinds, and the sheet behind a cell says which:

• Exact file data. Weight sizes are the bytes Hugging Face reports for the file you would download; machine memory and bandwidth are the makers’ specifications. Nothing is estimated.
• Derived from the architecture. The context cache is computed from the model’s config.json (attention design, layers, heads) and the context length: exact for a design we can read, the pessimistic classic formula for one we cannot (“architecture assumed conventional”).
• Fitted to measurements. Speed. Each chip’s effective bandwidth and fixed cost per token are fitted to the community llama.cpp table; the architecture and runtime terms are fitted to measured runs and judged on runs the fit never saw (section 6.1). Every fitted term carries the number of rows it rests on, and a term with too few rows is labelled assumed.
• Quoted. A hand-maintained build’s published speed. It is shown as a note with its source, never as the cell’s number, because the publisher named a memory size but not the chip, runtime or context it was measured on.
• Policy margins. The reserves for macOS and your apps, the working buffers and the MLX prompt scratch are safety margins chosen from owners’ observations, not measurements of your machine (sections 4 and 5). They are the numbers most worth arguing with, and section 8 says how far off they have been seen to be.

2. Words we use

• Model / weights — the file you download. Its size is the single biggest number in every fit calculation.
• Quantization (Q4, Q8, FP8, MLX 4-bit…) — storing the weights with fewer bits per number. Q4 ≈ 4.5 bits per weight and is the common local default; Q8 ≈ 8 bits and is near-lossless; FP8/BF16 are the “full” formats. Smaller quant = smaller file = faster, with some quality loss below Q4.
• Context window — how much text the model can hold “in mind” at once (your prompt + its answer + any documents). Measured in tokens; 1 token ≈ ¾ of an English word.
• Context cache (KV cache) — memory the model uses to remember the context while it works. It grows with the context length and is the part most fit charts ignore.
• Unified memory — on Apple silicon (and boxes like the DGX Spark), CPU and GPU share one pool. A model must fit in the part the GPU is allowed to use.
• Bandwidth (GB/s) — how fast the chip can read its own memory. It decides writing speed.
• Tokens per second (tok/s) — how fast the answer appears. Decode = writing the answer; prefill = reading your prompt before the first word appears.
• Active parameters — in mixture-of-experts (MoE) models only part of the model works on each token; that part is what gets re-read per word and what sets the speed.

3. Where the data comes from, and how fresh it is

How models get in. Every night a job lists what the five quantizer organisations above published or updated, finds the maker’s original repository for each (from the quantizer’s own “base model” note, otherwise by name), and reads that model’s config.json and file list. It keeps everything from the last twelve months that has at least a billion parameters and a thousand downloads; the ~25 featured models (hand-picked, current releases only) are always refreshed. Repositories that are edited variants — uncensored or “abliterated” builds, mixed-precision experiments, draft models — are skipped by name and logged. A model released today is normally in the tool tomorrow, with its real file sizes.

What can go wrong, and the safety net. Before new data goes live it is compared with what is live now: if a featured model lost its quantizations, a known file changed size by more than a tenth, a model’s architecture could suddenly not be read, or the catalog shrank, the update is held and a person looks at it. The page always shows the date of the data it is using. The full export (search index, one file per model, machines, benchmarks, factors, this document) is mirrored nightly to the public data repository.

4. How much memory is actually available for AI

Units first. Memory sizes are quoted in binary gigabytes: a “32 GB” Mac has 32 GiB, which is 34.4 GB in the decimal gigabytes file sizes use (× 1.0737). The tool converts the machine’s size before subtracting anything, so a 48 GB MacBook Pro is 51.5 GB, a 128 GB machine 137.4 GB, a 512 GB Studio 549.8 GB. The DGX Spark’s 128 GB is binary too.

Macs (Apple silicon). macOS does not let the GPU use all of the unified memory. By default it allows ⅔ on machines under 36 GB and ¾ from 36 GB: Metal’s recommended working set is logged at two thirds on 16 and 32 GB machines (10,922 and 21,845 MiB) and three quarters on 36, 48, 64 and 128 GB machines (28,991 / 38,655 / 49,152 / 98,304 MiB). Power users can raise this limit from Terminal (sudo sysctl iogpu.wired_limit_mb=…) — it works and it’s common, but it’s unsupported by Apple, so the tool treats it as an explicit toggle, off by default.

On top of that, the machine has to run macOS itself and — if it’s your everyday computer — your apps. We assume:

• macOS base: 6 GB with the default GPU limit. Once the limit is lifted, macOS is the only reserve left and on big machines it keeps more: the larger of 6 GB and 5.5 % of the memory (a 512 GB Studio keeps about 30 GB, which is what owners running with the override report). A policy margin.
• Your apps (the “I’ll also use this machine for work” toggle): a budget of 8 GB (light) / 16 GB (typical, the default when the toggle is on) / 24 GB (heavy) — a browser with many tabs alone is 5–7 GB. Adjustable in Advanced.

available = min( RAM × GPU-limit ,  RAM − 6 GB − apps )          default limit; RAM in decimal GB (nominal × 1.0737)
available = RAM − max( 6 GB , 5.5 % × RAM ) − apps                with the override

The min matters: on large machines the ¼ that macOS holds back already covers the OS and your apps; on small machines it doesn’t, and your apps become the real limit. Example: a 32 GB Mac mini (34.4 GB) used for work → min(34.4 × ⅔ = 22.9, 34.4 − 6 − 16 = 12.4) → 12.4 GB for AI; with nothing else running, 22.9 GB. That’s why a model that “fits 32 GB” on a spec sheet doesn’t fit a 32 GB machine you work on. Available memory by size, nothing else running: 16 GB → 11.2 · 24 → 17.2 · 32 → 22.9 · 36 → 29.0 · 48 → 38.7 · 64 → 51.5 · 128 → 103.1 · 256 → 206.2 · 512 → 412.3. With the override: 32 → 28.4 · 64 → 62.7 · 128 → 129.9 · 256 → 259.8 · 512 → 519.5.

Other platforms — see §7.

5. How much memory a model needs

needed = weights + context cache + working buffers + prompt scratch (MLX files above 8K of context)

• Weights = the published file size of the quantization you picked (not an estimate).
• Working buffers = 1.5 GB + 1 % of weights — activations and the runtime’s scratch space. A policy margin set from owners’ peak readings: about 1.1 GB above the file on a 17 GB dense model, about 0.4 GB above weights and cache on a 224 GB mixture-of-experts file (the earlier 5 % of weights rule charged that machine 11 GB).
• Prompt scratch (MLX files only) = 0.15 GB per 1K tokens of context beyond 8K. MLX allocates working memory to process a long prompt in one pass, and owners’ peak readings grow at about that rate on a 27B and a 397B model between 8K and 128K (3.6 GB at 32K, 18 GB at 128K); GGUF runtimes process the prompt in batches inside the buffers. A policy margin; it is its own segment in the memory bar and follows the file that runs, so a GGUF fix on an MLX column drops it.
• Context cache = bytes per token × context length. Bytes per token depends on the model’s attention design, which we read from its config:

bytes is the cache precision: FP16 by default (2 bytes), which is what llama.cpp, LM Studio, Ollama and MLX store unless told otherwise; 8-bit (1 byte) or 4-bit (0.5) selectable in Advanced. Most 2026 models use compressed or hybrid attention, so long context is far cheaper than older fit charts assumed. Models whose design we can’t read from the config are marked “architecture assumed conventional” and use the classic formula — the pessimistic choice.

Verdicts:

• Runs (solid dot) — needed ≤ 90 % of available.
• Tight (ring) — it fits, but with under 10 % headroom: 90 % < needed ≤ 100 %. Real sessions grow; treat it as “will work, watch it.”
• Runs with a compromise (ring) — the asked-for configuration doesn’t fit, but a change does, and the cell says which. The tool tries, alone and in combination: closing your work apps (only when the work toggle is on), the macOS memory-limit override (only on Macs, and only when the GPU limit is what binds), a shorter context (the next chips down: 128K → 32K → 8K), a smaller quantization of the same format, and any hand-maintained special build (below). Every combination that fits gets a cost — closing apps 1 · the override 2 · each context step 4 · a smaller quant 8 (9 under 3 bits) · a special build 16 · tightness ½ — and the cheapest wins; ties go to the higher-quality (more bits) and longer-context option. Up to three genuinely different alternatives are kept for the tooltip (“or: at 32K context instead of 128K”).
• Doesn’t fit (dash) — nothing allowed fits at any context. The cell still explains itself: “the smallest allowed build, UD-IQ2_XXS (2-bit) at 711 GB, needs about 724 GB at 128K; this machine offers 520 GB at most.”

The quality floor. The automatic search never proposes a build under 2 bits per weight, judged by the file’s label (IQ1_* / Q1_* are out; IQ2_*, Q2_*, MLX-2bit are in). The label is used rather than the measured bits-per-weight because dynamic quantizations keep some tensors at higher precision — a 1-bit-labelled build of a large mixture-of-experts model reads as 2.3 effective bits. When only a sub-floor build would load, the dash says so (“a 1.3-bit build (466 GB) would load, quality unknown”); Advanced lets you lower the floor and pick such builds by hand.

Special builds. A few community builds are not plain quant files — Qwen 3.8 Flash Next’s 4-bit build keeps 45.8 GB in memory and pages its 51 GB n-gram table from SSD. Those are hand-maintained entries (resident size, source, and the measurement that justifies them) and appear in the search as their own compromise: “with the SSD-paged build (45.8 GB resident)”. Their speed is not estimated: the tool has no per-token read to reason from, and the publisher’s measurement (36 tok/s on a 64 GB MacBook Pro) names a memory size but not the chip, runtime or context, so it cannot be carried to any catalog machine. The sheet quotes it as a note with its source and shows no speed figure for the build.

6. Speed

6.1 Writing speed (decode) — the number that matters day to day

To write each token, the machine re-reads the model’s active weights plus the context cache, and then does a fixed amount of work that does not shrink with a smaller file: launching the kernels of every layer, routing between experts, attending over the context. The time per token is the sum of the two, and the speed is its inverse:

GB per token = active_params × bytes per weight (quant) + cache bytes per token × context length
ms per token = read + fixed + context
    read     = GB per token ÷ effective bandwidth × 1000 × read factor   (K-quant GGUF files on Apple silicon 1.25, everything else 1)
    fixed    = (chip fixed cost + architecture cost) × MLX factor         (GGUF files 1; section 6.2)
    context  = (attention cost + MLX attention cost) × context ÷ 32K
tok/s        = 1000 ÷ ms per token

Until version 0.8 the tool used bandwidth × efficiency ÷ GB per token, one efficiency factor per chip. That factor was calibrated on a 3.8 GB read (Llama-2-7B at Q4_0) and carried the fixed cost inside it, so it was wrong in both directions away from that size: models with a small active set (a 2 to 4 GB read per token) came out 1.5 to 3 times too fast, and 8-bit dense models on the Ultra chips 0.55 to 0.85 times too slow. The fixed-cost model separates the two.

Chip profiles are fitted, not assumed. For every chip, calibrate.py takes the F16, Q8_0 and Q4_0 rows of the community llama.cpp table (13.5, 7.2 and 3.8 GB reads of the same 7B model) and fits a straight line of milliseconds per token against gigabytes read: the slope is the effective bandwidth, the intercept the chip’s fixed cost. The effective bandwidth is stored as a share of the chip’s specification, because that share carries across the GPU bins of one chip (an M4 Max 32-core at 410 GB/s fits 0.88, the 40-core at 546 GB/s 0.90; an M3 Max 30-core 0.96 against the 40-core’s 0.98) while the GB/s does not.

Two-row chips (M1, M3, M5, the Spark) have a line through two points and no slack; the rest fit within 2 % except the M1 Max (8 %, six rows from several machines) and the M5 Max (6 %). The Ultra chips read at 80 to 88 % of their specification: the multi-die design pays a consistent price. Tier medians stand in for chips without rows and are labelled “estimate — unmeasured”: base 0.94 / 5.8 ms, Pro 0.95 / 5.8 ms, Max 0.91 / 5.1 ms, Ultra 0.83 / 5.2 ms; M6 and M6 Pro take the base and Pro medians. The M5 Ultra is the one explicit assumption: 85 % of its 1,200 GB/s (the Ultra tier’s share) and 1.5 ms of fixed cost (between the M5 Pro’s 2.6 and the M5 Max’s 0.5), until a measured row exists. The chip name is matched without its GPU bin (M5 Max (40-core GPU) → M5 Max); a platform without a profile (AMD ROCm boxes) gets no speed number rather than an invented one.

Architecture cost is the extra fixed time a model’s design adds per token, on top of the chip’s own. The class comes from the config: a dense model adds nothing; a mixture of experts adds its routing; one with linear-attention layers (the Qwen 3.5, 3.6 and 3.8 mixtures, Nemotron 3) adds those layers’ recurrent state; one with compressed (latent) attention (GLM-5.x, Kimi, DeepSeek-R1) adds the decompression; DeepSeek V4 is its own class, set by hand in the overrides file, because its FP4 experts and 512-wide attention measure far above the others. Each cost is the median over the measured GGUF rows of its class at short context, after the read and the chip’s fixed cost are subtracted:

Read factor. On Apple silicon, GGUF K-quant and IQ files (Q4_K_M, UD-Q4_K_XL, IQ2_M …) read about 25 % slower than the plain Q4_0, Q8_0, F16 and MXFP4 formats, because their dequantization is heavier on Metal. One dense row measures it (1.25, kept as an assumption until more rows exist), and it applies only on Apple silicon: no CUDA row measures it, so the Spark reads K-quants at the plain rate. MLX files read at the plain rate.

Context cost. Beyond the cache re-read (already in GB per token), attending over a long context costs time that does not show up in the bytes: 5 ms per token per 32K of context for latent-attention models (an assumption; see section 8 on the rows that could have fitted it), nothing for grouped-query attention, and on MLX a cost that depends on the chip generation: 6.4 ms per 32K on the M1 to M4 chips (fitted on 14 rows of the M3 Ultra and M1 Ultra sweeps to 128K) and 0.8 ms per 32K on the M5 generation (fitted on 4 rows of the oMLX M5 Max ladders to 64K; section 6.2). Because the cache is also in the read, speed visibly falls as the context grows: more for classic-attention models, less for hybrids, and more on the M1 to M4 chips than on the M5.

How the fit is judged. The terms above are fitted on the training split of validation.json (73 measured runs from the llama.cpp maintainers and eleven community contributors, each row with its source, grade and context length) by coordinate descent on medians, and judged on a held-out split the fit never saw: every Gemma 4 row and every row from the omlx benchmark site, split by submission so that a contributor’s sweep is never half in each. On the held-out rows the median error is a factor of 1.14 (90th percentile 1.29); the multiplier model scored 1.21 (90th percentile 2.66) on the same rows. On the training rows: 1.06 against 1.33. The release rule was a held-out median under 1.25 and no measured row getting worse than the multiplier by more than 1.30; both held. factors.json carries every term with its row count, range and source, and calibrate.py --report regenerates the row-by-row comparison. Chips whose profile is a tier median are flagged in the sheet.

6.2 Runtime: GGUF (llama.cpp / LM Studio) vs MLX

The baseline above is llama.cpp with a GGUF file. Apple’s MLX runtime (MLX files, used by LM Studio’s MLX engine, Ollama’s MLX backend, mlx-lm) changes the fixed cost per token, not the read: the measured gains sit where the fixed cost dominates (a 3B-active mixture of experts on an M4 Max: 71 → 130 tok/s) and vanish where the read dominates (8-bit files and big dense models: 0.96 to 1.2×), and on the M1 generation MLX is slower on mixtures (0.8×). So the tool multiplies the fixed cost by a factor per model group, fitted on the MLX rows of the training split:

Also: MLX files are 5–13 % smaller for the same nominal quant (the tool uses the actual MLX file size when you pick MLX), and MLX prefill is 30–40 % faster. Long context: on top of the cache re-read, MLX pays a cost per token per 32K of context that depends on the chip generation. On the M1 to M4 chips it is about 6.4 ms (fitted on the Qwen 32B and Qwen3.5-397B sweeps on the M3 Ultra and the Qwen 27B rows on the M1 Ultra, to 128K), which is why llama.cpp with flash attention catches up and passes MLX there somewhere past 30K tokens (on an M3 Ultra, MiniMax-M2.1 at 146K: MLX 6 tok/s against llama.cpp 12). On the M5 generation it is about 0.8 ms (fitted on the oMLX M5 Max ladders of Qwen 3.8 27B to 64K and Nemotron 3 Super at 32K, accelerations off): the same model that loses 55 % from 4K to 64K on the M3 Ultra loses 23 % on the M5 Max, and the difference beyond the cache read is what the M5 GPU’s neural accelerators remove. A chip whose generation the tool cannot read (a custom machine) is charged the pre-M5 cost. The M5 Ultra inherits the M5 value unmeasured, like the rest of its profile. The crossover falls out of the terms instead of the hard-coded 24K–36K fade of earlier versions.

6.3 Acceleration: MTP and draft models (DFlash / DSpark)

Some models can write several tokens per step. Two ways: MTP (multi-token-prediction heads built into the model — we detect them in the config) and draft models (a small helper model proposes a block of tokens the big model verifies in one pass; DFlash and DSpark drafts exist for many popular models — we find them on Hugging Face daily). Output is identical; only speed changes. The tool offers these only for models that actually have them, and shows the result as a range beside the baseline, never as the headline:

These ranges are computed by calibrate.py from the paired rows in benchmarks.json (each accelerated row records the unaccelerated speed of the same setup), so they move as measurements are added.

Gains depend on the task (code and math accept longer drafts than chat) and on quant (8-bit targets accept more than 4-bit).

6.4 Reading speed (prefill) and time to first answer — v1.5

Prefill is limited by compute, not bandwidth. We calibrate it per chip from the measured pp512 rows of the same community table (M5 Max: 3,220 tok/s on a 7B model vs 886 on M4 Max — Apple’s neural accelerators are measurable), scaled by active parameters, and show time to first token for 8K / 32K / 128K prompts. Chips without measured prefill rows are labeled estimates.

6.5 “Feels like” — the translation next to every speed

7. Other machines: the same math, different facts

One sentence to remember: memory adds up across cards and boxes; speed doesn’t.

8. Known limitations (read before arguing with a number)

1. Everything speed-related is an estimate unless a measured row exists for exactly your chip, model, quant and runtime — then we show the measurement next to the estimate.
2. Fits ≠ runs well. A model that loads at 2-bit, or crawls at long context, is running the way a car runs in first gear. Rings tell you the compromise.
3. Sustained load is real. Long inference sessions run the GPU flat out; laptops get hot and may throttle. The tool doesn’t model thermals.
4. The macOS limit override is unsupported by Apple; the tool defaults to the official behavior.
5. Quantization quality varies by quantizer and method (dynamic “UD” quants keep more accuracy at the same size); we show quantizers’ own quality notes where published and don’t invent scores.
6. MoE active parameters are computed from the config; makers’ published figures take precedence when they differ.
7. Models newer than the data date aren’t in the tool yet; custom entries are the workaround.
8. “Architecture assumed conventional” entries use the pessimistic classic cache formula until someone tags the architecture.
9. Quick custom models (name, parameters, file size) get an assumed design: about 32 × (parameters ÷ 8B)^0.4 layers with 8 KV heads of 128, all caching. Enter the real fields in the full form for exact context math.
10. Two-point chip profiles. The M1, M3, M5 and the DGX Spark have only two dense rows, so their line has no slack and no error estimate; the Spark’s rows are the maintainers’ own, the Macs’ come from one community table.
11. Thin classes and one-row terms. The linear-attention and latent-attention mixture costs rest on two rows each, DeepSeek V4 on three (range 13 to 33 ms); the K-quant read factor and the MLX factor for small mixtures on M4 and newer rest on one row each and are labelled assumed; the latent-attention context cost is an assumption. These are the terms a single new measurement can move.
12. The held-out rows cover no latent-attention mixture and no mixture on the Spark, so the 1.14 median is a statement about dense models, Gemma’s mixtures and the Spark’s small dense models; the oMLX M5 Max submissions that run past 8K moved from the held-out set to the training set in 0.10, because they are the only long-context M5 rows there are, which leaves 13 held-out rows. Every row came from a public post; none was measured under controlled conditions.
13. Rows the model gets wrong by 2× or more. MiniMax-M2.1 at 30K and 146K of context (mlx-lm issue 763) runs at half the model’s prediction on both runtimes; Kimi K2.6 at 10K and 20K on a 512 GB Studio with a 440 GB build runs at a third and a fifth. Both look like memory pressure and are kept as report-only rows; a long-context term that catches them needs cleaner measurements.
14. Memory margins are policy, and their residuals are known. Against owners’ peak readings, with every reading converted to decimal GB where its unit is known (oMLX reports binary gigabytes), the tool over-estimates by 0.5 to 1.4 GB on 17 to 30 GB dense GGUF files, by 0.3 to 1.7 GB on a 27B MLX file from 4K to 64K, by 1 to 5 GB on a 224 to 422 GB MLX file, and under-estimates a 31B GGUF file at 37K by 2.4 GB. Readings whose unit or method is not stated are kept as published and flagged in validation.json. Read a tight verdict with that in mind.
15. Active parameters are the makers’ published counts where they exist. The config-derived estimate is the fallback, and it has been wrong by 2× on models whose configs hide the expert width (MiniMax M3) or use a latent expert dimension (Kimi K3); the overrides file carries the published figure and the reason. Every validation row must use the catalog’s own count and cache bytes (a test enforces it), so a passing fit says something about the live path.
16. The M5-generation MLX context cost rests on four rows of two models on one machine. The 0.8 ms per 32K comes from the oMLX M5 Max ladders (Qwen 3.8 27B at 16K, 32K and 64K, Nemotron 3 Super at 32K), one contributor, accelerations off, and it is applied to every M5 and later chip, including the M5 Ultra, which nobody has measured yet. The pre-M5 value rests on the M3 Ultra and M1 Ultra alone. A long-context ladder on an M4 chip, or on an M5 Ultra once it ships, is the measurement that would settle which generation each value really belongs to.

9. Worked examples

All numbers below come from the live data (file sizes read from Hugging Face, factors from factors.json) and are reproduced by the engine’s tests.

A. 32 GB Mac mini (M6), Qwen 3.8 27B at Q4 (LM Studio’s Q4_K_M, 16.8 GB), 32K context. The machine has 34.4 GB in file-size units. Available with the default GPU limit = min(34.4 × ⅔, 34.4 − 6 − 0) = 22.9 GB. Needed = 16.8 (weights) + 2.2 (cache: 64 KB/token at FP16 × 32K, plus the hybrid layers’ constant state) + 1.7 (buffers: 1.5 GB + 1 %) = 20.7 GB → tight at 90 %, a ring. Speed ≈ 6.5 tok/s (“usable”; the M6 has no measured row yet, so it uses the base-tier profile and carries the estimate label). Turn the work toggle on: available = min(22.9, 34.4 − 6 − 16) = 12.4 GB → the ring says “runs if you close your apps; tight”; the tooltip’s alternative is a 2-bit build that fits with the apps open.

B. 256 GB Mac Studio (M5 Ultra), GLM-5.3 Flash at Q4 (UD-Q4_K_XL, 199.7 GB), 128K context. Available with the default GPU limit = ¾ of 274.9 GB = 206.2 GB; needed = 199.7 + 1.5 (cache — compressed attention on 11 of 45 layers, FP16) + 3.5 (buffers) = 204.8 GB → tight at 99 %, a ring. With the override, available = 274.9 − max(6, 15.1) = 259.8 GB → runs at 79 %; with the work toggle on as well (243.8 GB) it runs at 84 %. Speed on the M5 Ultra (assumed profile: 1,020 GB/s effective, 1.5 ms): 18B active × 4.97 effective bits ≈ 11.2 GB read per token at 8K → 13.8 ms read + 1.5 ms chip + 11.7 ms latent-attention mixture + 1.3 ms context ≈ 28 ms → ≈ 35 tok/s at 8K, ≈ 31 at 32K and ≈ 21 at 128K; with MLX the fixed part shrinks to 0.77× (≈ 39 tok/s at 8K); with its MTP heads, more again. The same model on a DGX Spark: 129.4 GB available after the 8 GB reserve; Q4 does not fit; the 3-bit build (UD-IQ3_XXS, 120.4 GB) fits tight at 96 %, at ≈ 15 tok/s (the Spark’s 258 GB/s effective and 2.6 ms, plus the same 11.7 ms architecture cost), and the 2-bit build (UD-Q2_K_XL, 108.7 GB) is the alternative at ≈ 16 tok/s.

C. 512 GB Mac Studio, GLM-5.2 at Q4 (UD-Q4_K_XL, 467.3 GB), 128K context, override on. Needed = 467.3 + 11.8 (cache — compressed attention, FP16) + 6.2 (buffers: 1.5 GB + 1 % of 467) = 485.2 GB; available = 549.8 − max(6, 30.2) = 519.5 GB → tight (93 %). Without the override (available 412.3 GB) the cheapest fix is the override itself; the alternative is the 3-bit build (UD-IQ3_S, 309 GB), which runs comfortably.

D. 256 GB Mac Studio, GLM-5.2, 128K. A ring: with the override (259.8 GB available) the 2-bit build UD-IQ2_M (238.6 GB) fits with its cache and buffers (254.2 GB) at 98 %, tight; the alternatives are the smaller 2-bit UD-IQ2_XXS, also tight, and a 32K context. Without the override nothing fits, so the fix is the override plus the 2-bit build.

E. 512 GB Mac Studio, Kimi K3 (2.8T parameters). A dash: the smallest 2-bit build is 711 GB and needs about 724 GB, against 520 GB at most with the override. Its 1.3-bit build (466 GB) would load at 92 % with the override, tight — quality unknown, and only if you lower the floor in Advanced.

F. 64 GB MacBook Pro (M5 Max), Qwen 3.8 Flash Next. The machine has 68.7 GB; three quarters of it is 51.5 GB. Every regular 2-bit-and-up build starts at 75 GB, so none fits. The hand-maintained SSD-paged build (45.8 GB resident) fits without the override: 45.8 + 0.9 + 2.0 = 48.6 GB → ring: “with the SSD-paged build; tight” (94 %), no speed figure: the publisher’s 36 tok/s on a 64 GB MacBook Pro is quoted as a note, because the chip, runtime and context of that measurement were not stated.

G. The buying map for Qwen 3.8 27B at Q4 (Q4_K_M), 32K, at least 12 tok/s. Every current configuration is evaluated as in A. With no minimum speed the cheapest that fits as asked is the 32 GB Mac mini (M6) at $1,299 (tight, ≈ 6.5 tok/s, estimate); at 12 tok/s or more it misses on speed, and the pick becomes the 48 GB Mac mini (M5 Pro) at $2,299 (≈ 13 tok/s, comfortable, 18 GB to spare), with the 64 GB M5 Pro mini ($2,699, 31 GB to spare) as the alternative with headroom; a laptop preference makes it the 48 GB MacBook Pro (M5 Pro) at $3,099. Under a $2,000 budget at 12 tok/s nothing qualifies: the card names the 48 GB mini as $299 over the budget, the 32 GB mini as too slow (≈ 6.5 tok/s), and the 24 GB M5 Pro mini as qualifying only with a change (the memory-limit override and 8K context). The 512 GB Studio fits at every setting but has no list price yet, so it is listed as qualifying without one and is never the pick.

10. Changelog

• 0.10.2 — 2026-09-05. The buying map’s speed axis is numbered at the feels-like band edges (0, 5, 12, 30, 60 and the end of the axis) instead of round ticks, so every band’s border carries its number; the “cheapest that qualifies” and “yours” tags are measured with their labels, so a tag can no longer run into the memory-size column. No formula or constant changed.
• 0.10.1 — 2026-09-05. The tool went public: the tool and this page are indexed, in the site map and in the site’s navigation (a phone menu was added for it). No formula or constant changed.
• 0.10 — 2026-09-05. The MLX long-context cost is now per chip generation (sections 6.1 and 6.2): 6.4 ms per token per 32K on the M1 to M4 chips as before, 0.8 ms on the M5 generation, fitted on the oMLX M5 Max ladders. The old single value, fitted only on M3 Ultra and M1 Ultra sweeps, charged the M5 chips a penalty they do not show: the tool under-predicted the M5 Max by 17 to 21 % at 32K to 64K on MLX, and the M5 Ultra by about 45 % at 256K, so the Ultra-to-Max ratio at long context read 1.4× where the bandwidth ratio says 1.8×. The two oMLX M5 Max submissions that run past 8K moved to the training split to fit the new term (holdout 13 rows, median ×1.14, gate passed; train 79 rows, median ×1.05). Section 8 gains item 16 on how thin that fit is. Numbers at 32K and below, and every GGUF number, do not move.
• 0.9.6 — 2026-09-05. The buying map after the founder’s first look: the “doesn’t fit” gutter is gone (a dash just left of the zero line, never on the speed axis); machines at the same speed get a label each, one under another (a joined label only past three or where nothing fits beside them); a legend above the map explains the five marks; the answer card and the list under the map are retired, the map itself names the cheapest qualifying configuration and the machine you own, and a single line says when nothing qualifies. The cell sheet is in sections: the list price first, the memory bar without the sentence that repeated it, the speed as a number with the feels-like phrase and the acceleration ceiling, the estimate’s provenance on hover and on demand, the sources on one line. No formula or constant changed.
• 0.9.5 — 2026-09-05. “Which machine?” is now a buying map (section 1): every current machine configuration placed by memory size and estimated writing speed for one model at one build and context, a Q4 / Q8 / own-build switch on a shared axis, a minimum-speed line, and three new settings (a budget, laptop or desktop, the machine you own) that turn the marks into a recommendation with the rules written down: fits as asked, speed at or above the minimum, within the budget, matching the preference; a compromise is a ring at its fix’s speed and never qualifies; a machine that does not fit is never placed at a speed; an unpriced machine is never called cheapest; nothing qualifies is said plainly with the nearest misses. Worked example G. The share link gains the budget, the preference and the owned machine; links made before the map open it at its defaults. No formula or constant changed.
• 0.9.4 — 2026-09-05. “What can it run?” tidied after a second look: the data columns are spaced apart and a row says one thing per column (a dash reads “Doesn’t fit”, the reason moves to the tooltip and the sheet); the whole row opens the sheet; size bands are headed in full (“40 to 120 GB at Q4 · 6 models”); the expanded table has a proper header (“Build” over the rows, “Context window” over the columns) and vertically centred row labels. No formula or constant changed.
• 0.9.3 — 2026-09-05. “What can it run?” reworked after the founder’s first look: models are grouped by the size of their Q4 file instead of the catalog’s laptop / studio / frontier tiers; every row carries the parameter count and the memory the configuration in the row needs; the expanded view is a full-width table per model (rows = builds, columns = context windows, speed and memory under each mark) under a tinted header band that repeats the model’s summary, with a legend above the list. No formula or constant changed.
• 0.9.2 — 2026-09-05. The fifth tile, “What can it run?”: one machine at one memory size against the whole catalog, best build first, expandable into every build and context (section 1). Any catalog machine can be picked, including older ones and the machines typed into Advanced; the assumptions in Settings apply as they do everywhere; a cell opens the same sheet, where only the global fixes (closing your apps, lifting the limit) can be applied because a build or context is a cell of the grid, and “Which machine do I need?” adds the model to the table at that build and context. The share image and presentation mode show the table for this view. A “Show benchmarks · coming soon” chip records interest through the notify card. No formula or constant changed.
• 0.9.1 — 2026-09-05. Corrections from a second independent audit of the 0.9 release. A hand-maintained build’s published speed is no longer used as the cell’s number on any machine (it had been shown on every Apple machine with a 64 GB option); it is quoted as a note with its source, and the sheet’s file link now follows the build that fits. Active parameters: MiniMax M3 23B (was 49B, the config hides the expert width), Kimi K3 104B (was 165B, the experts work on a 3,584-wide latent), Qwen3-Coder-Next 3B (was 4.0B), Gemma 4 E4B 4.5B effective (was 8B: the per-layer embedding tables are lookups), each from the model card; the parser now reads a latent expert width and Gemma’s embedding tables itself. Validation data: four rows transcribed from the wrong context of one source corrected, memory readings now carry their published unit and metric with oMLX’s binary gigabytes converted (the 27B MLX residual at 64K is 1.7 GB, not 3.7), the file sizes of the mixture-of-experts memory rows added so the memory report covers every row, three readings taken at a configured rather than filled context moved to report-only, and a test that refuses a validation row whose active count or cache bytes differ from the catalog record. The re-fit moved no term: held-out median ×1.14, gate passed. Section 8: item 14 corrected, item 15 added.
• 0.9 — 2026-09-05. The speed model is now a fixed-cost model: milliseconds per token = a read of the bytes per token at the chip’s effective bandwidth + a fixed cost (the chip’s, fitted from the F16/Q8_0/Q4_0 rows of the community table, plus an architecture-class cost fitted on measured runs) + a context term; tok/s is its inverse (section 6.1). MLX now scales the fixed cost by model group instead of multiplying the speed, GGUF K-quant files on Apple silicon carry a 1.25 read factor, and the MLX long-context penalty is a fitted term instead of a 24K–36K fade (section 6.2). Fitted on 73 measured runs and judged on 19 held-out ones: median error ×1.14 against ×1.21 for the multiplier model, with no row getting worse. What moves: mixtures of experts with a small active set lose the 1.5–3× the old model gave them (GLM-5.3 Flash on the M5 Ultra ≈ 35 tok/s, was 59; its 3-bit build on the Spark ≈ 15, was 25), 8-bit dense files on the Ultras gain, and the Mac-vs-Spark contrast on GLM-5.3 Flash shrinks from about 4× to about 2.3×. Memory margins became explicit policy (sections 4 and 5): buffers are 1.5 GB plus 1 % of the weights instead of 5 % capped at 16 GB (a 467 GB file is charged 6.2 GB, not 16), MLX files carry 0.15 GB of prompt scratch per 1K tokens beyond 8K as their own bar segment, and with the GPU limit lifted macOS keeps the larger of 6 GB and 5.5 % of the memory (30 GB on a 512 GB Studio). New subsection in section 1 on the five kinds of number. Worked examples rewritten: A is tight at 90 % (the buffers grew), B is tight at 99 % without the override, C at 93 %, E’s nearest miss is 724 against 520 GB. The aired-chart cross-check disagrees in 17 of 108 cells (the realistic chart’s 48 GB Qwen cell is tight at 92 %). Data: validation.json (120 measured speed and memory rows with sources, grades and the train/holdout split) joins the mirrored files; benchmarks.json gains the F16 and Q8_0 rows per chip (role floor-fit, outside the retired multiplier’s medians).
• 0.8 — 2026-09-05. Verified defects from two independent audits against measured runs. Memory sizes are now converted from the binary gigabytes Apple and NVIDIA quote to the decimal gigabytes file sizes use before any reserve is subtracted (32 GB = 34.4 GB, 512 GB = 549.8 GB); the macOS GPU limit is ⅔ under 36 GB and ¾ from 36 GB, as Metal’s logged working set shows, not ¾ from 64 GB (36 and 48 GB Macs gain about a fifth of their usable memory). The context cache defaults to FP16, which is what every runtime stores unless told otherwise (it was 8-bit); Gemma 4’s shared key/value projection no longer halves the stored cache (26B-A4B ≈ 1.4 GB at 8-bit, 2.9 GB at FP16, was 0.7); a hand-maintained build’s measured speed is shown only on the machine class it was measured on. DGX Spark efficiency is 0.75 from five llama.cpp maintainer rows (build b7941); it was 0.40 from a single row that was recorded as a 40 GB Q4 llama.cpp run but was in fact LMSYS’s Llama 3.1 70B at FP8 under SGLang (kept in the data for provenance, excluded from calibration) — every Spark speed rises about 1.9×. Mistral Small 4’s active parameters are 6.5B per its model card (the config-derived estimate was 8.0B). Worked examples rewritten: the 256 GB Studio fits GLM-5.2’s 2-bit build with the override (D is a ring), the 64 GB MacBook Pro runs Flash Next’s SSD build without the override (F), the Spark fits GLM-5.3 Flash’s 3-bit build (B). The aired-chart cross-check now disagrees in 16 of 108 cells, none from a formula error. The MLX fade end is single-sourced at 36K (the data file said 30K).
• 0.7 — 2026-09-04. The memory chart became “How much memory?”: absolute bars of what a configuration needs (weights + context cache + buffers), machine-independent, so the same model at two builds or two context lengths can be compared; one machine’s limit is a dashed reference line. No formula or constant changed.
• 0.6 — 2026-09-03. The simple UI: all configuration moved into a settings panel (machines with tickable memory sizes, models as one-build-one-context columns, drag to reorder, the two switches, Advanced folded in); four chart tiles (fit table, speed, which machine, memory); the memory breakdown became a full chart; a Feedback button and a closable notify card replaced the forms under the table. Section 1’s reading guide updated. No formula or constant changed.
• 0.5 — 2026-09-03. The About page (/tools/llm-sizer/about) renders this document with a FAQ and the sources behind every machine, benchmark row and factor; every number in a cell’s sheet now links to its source (section 1). Correction recipe by file (section 11). Colours inside the tool darkened for contrast (the video’s palette stays on the filming canvas). Cross-check against the charts that aired in the Apple-Macs video (2026-08-29), reproduced through the tool’s own table path and kept as a test: the tool disagrees in 14 of 108 cells, none from a formula error. The reasons: the chart drew “does not fit at Q4” as a dash where the tool shows the cheapest compromise as a ring (a 2-bit or 3-bit build, a shorter context, the memory-limit override, or closing your apps in the realistic chart); the chart assumed the three-quarters memory limit on a 32 GB Mac where macOS allows two thirds (Qwen 3.8 27B at Q4 + 128K needs 22.2 GB against 21.4); Qwen 3.8 Flash Next’s Q4 file is 119 GB (the n-gram table is inside it), not the ≈ 85 GB the chart assumed; GLM-5.3 Flash at Q4 needs the override on a 256 GB Studio once buffers count; GLM-5.2’s 2-bit build does not fit 256 GB with its cache and buffers, and on 512 GB it needs the override and lands at 97 % (tight, a ring); and the speed table’s flat 0.48 efficiency is now per chip (section 6.1). No formula or constant changed.
• 0.4 — 2026-09-03. The app. New “How to read the table” subsection (section 1): the three marks, the two switches, tight cells as rings, speed shown for the configuration that fits, links and custom entries. Fix descriptions now separate the tight note with a semicolon instead of a dash; the alternatives list never contains a superset of an earlier alternative. No formula or constant changed.
• 0.3 — 2026-09-02. The engine. Verdicts now have four states (runs / tight / compromise / doesn’t fit) with the fix search’s costs written down; the 2-bit quality floor (by label) and hand-maintained special builds added; MLX factor fades 24K → 36K instead of dropping at 30K; efficiency computed on active weights for mixture-of-experts benchmark rows (Ultra tier 0.44, M5 Ultra assumption 0.55); chip names matched without their GPU bin; ROCm machines get no speed number. Worked examples rewritten from the live data: Qwen Q4 is 16.8 GB (LM Studio’s file), GLM-5.3 Flash ≈ 59 tok/s at 8K on the M5 Ultra, its Q4 does not fit the DGX Spark (the 2-bit build runs ≈ 17 tok/s), GLM-5.2 on 256 GB is a dash once buffers are counted, Kimi K3 and Flash Next examples added.
• 0.2 — 2026-09-02. Data pipeline built. Section 3 now describes how models are discovered (five quantizer organisations, twelve-month window, featured list, name-based exclusions) and the publish-time safety net. Efficiency constants are now generated by calibrate.py from benchmarks.json into factors.json (tier medians 0.79 / 0.74 / 0.63 / 0.45 / 0.40; measured chips use their own value, M5 Max 0.82); M5 Ultra documented as an explicit 0.57 assumption. Acceleration ranges computed from paired rows. Gemma 4 cache corrected to its global-layer heads (≈ 0.7 GB at 128K, not 1.3). Machines catalog: one row per bandwidth bin (M5 Max 460 vs 614 GB/s; M6 mini 153 GB/s at 16 GB vs 170 at 24/32 GB). Spark speed in example B corrected from ~14 to ~11 tok/s.
• 0.1 — 2026-09-02. First draft, written from the Phase 0 research spike: efficiency by chip tier from the community llama.cpp table (M1–M5 Max), context-cache formulas by attention design read from model configs, MLX runtime factors from 2026 community reports, MTP/DFlash factors from MTPLX and mlx-dspark measurements.

11. How to correct us

Use the feedback form in the tool (it attaches your exact configuration) or open an issue on the open-data repository, ideally with: machine, model, quant, context, runtime, and the number you measured. Accepted measurements go into the calibration data with your handle as the source, and the relevant row above changes with a changelog entry.

File by file (the canonical copies of machines.json, benchmarks.json and models.overrides.json live in the tool’s repository and are mirrored to the open-data repository every night; propose a change as an issue or pull request there and it lands in the tool):

• A machine fact (memory options, bandwidth, price, status): edit the row in machines.json and add the page you are citing to its sources.
• A speed you measured: add a row to benchmarks.json with chip, bandwidth_gbs, memory_gb, model, quant, weights_gb (decimal GB; active_weights_gb too for a mixture-of-experts model), runtime (llama.cpp-gguf, mlx, or another runtime’s name — only llama.cpp rows calibrate the efficiency), accel (none, mtp, dflash, dflash2, dspark), tg (writing tokens per second), pp (prefill, optional), source (a public link), date, and where you have them build (the runtime build) and context_tokens (tokens already in context when you measured); accelerated rows also carry baseline_tg, the same setup without the acceleration; rows meant for the chip-profile fit (F16, Q8_0 and Q4_0 reads of one dense model at short context) carry role: "floor-fit". A measurement of a whole configuration (a model at a quant, runtime and context, with tokens per second or peak memory) goes into validation.json with a grade (A: the maintainers’ or a reproducible tool’s number; B: a community post with the setup stated; C: a headline number) and a split (train, holdout, or report for rows the fit must not learn from); a memory reading also records the number as published, its unit (binary or decimal gigabytes, or unknown) and what was measured (allocator peak, process footprint, a configured context), and the row’s active count and cache bytes must be the catalog’s. Do not edit factors.json: it is regenerated from the rows.
• A model fact the Hugging Face API cannot express (a published active-parameter count, a licence threshold, a hand-maintained build): models.overrides.json.
• A formula or constant: open an issue. FORMULAS.md is a nightly copy of this document and is not edited in the data repository.