First measured speeds for Flash-Next Optimized Quality — M3 Ultra, 256 GB (D2 beats the D3 default)

[Thomas-Kluge]

The README says of Qwen3.8-Flash-Next-MTPLX-Optimized-Quality: "It has not yet been run on a 256 GB Mac, and its speed has not been measured." Here is that run: a Mac Studio M3 Ultra with 256 GB, first community numbers for the pack, and first M3-family numbers of any kind on this board, per the "no M1–M4 numbers of its own" note on mtplx.com/benchmarks.

Machine and versions

Hardware	Mac Studio (Mac15,14), Apple M3 Ultra, 256 GB unified memory
macOS	27.0 (26A428)
MTPLX	2.12.0 (app + CLI)
MLX / mlx-lm	0.32.2 / 0.31.3 (stock)
Pack	Youssofal/Qwen3.8-Flash-Next-MTPLX-Optimized-Quality (169.96 GB download)

Command

mtplx tune --model Youssofal/Qwen3.8-Flash-Next-MTPLX-Optimized-Quality --retune

Fans pinned by the tuner (baseline ~1004 RPM, held ~3350 RPM against a 3625 RPM target), burst lane: 183-token warm coding prompt, 512 tokens generated, sampled at the model's own contract (temperature 1.0, top-p 0.95, top-k 20), single stream, thinking as resolved by the preset.

Results

Mode	Decode tok/s	vs AR	Acceptance
AR	39.9	1.00x	—
D1	76.7	1.92x	0.963
D2	95.3	2.39x	0.984 / 0.944
D3	87.2	2.18x	0.948 / 0.866 / 0.799

Verdict written by the tuner: mtp_depth_wins, saved depth 2 for this model.

Observations

1. Depth 2 beats the shipped depth-3 default on this machine by ~9% (95.3 vs 87.2). The acceptance curve explains it: the third draft token only lands 0.799 of the time, so the extra draft cost stops paying. --retune picked correctly; it might be worth noting next to the "This family accepts up to depth 5" line that depth 5 is not automatically best and the M3 Ultra already lands on 2 in the burst lane.
2. The 8-bit head accepts better than the published 4-bit numbers. The 27B Optimized Speed launch note reports acceptance 0.95 / 0.88 / 0.80 by depth; here D1/D2 run 0.963 and 0.984/0.944 and even D3's first slot is 0.948 — consistent with the quality pack's higher-fidelity drafts.
3. AR at 39.9 tok/s against 95.3 MTP is a 2.39x speculative multiplier, in line with (slightly above) the 1.6–2.24x range quoted elsewhere, on a chip one generation behind the author's M5 Max.
4. Memory: the server loaded the full 8-bit body with room to spare on 256 GB at the default context; no swap or memory-gate events during the four isolated candidate runs. (No exact peak figure was captured by the tuner; happy to measure one on request.)

Raw artifacts (per-candidate JSON with full accept/verify telemetry, logs, and tune.json):
https://gist.github.com/Thomas-Kluge/9779e5ff922eb4980ab577a32f6e8f95
(tune.json, ar.json, d1.json, d2.json, d3.json and the four run logs, from
~/mtplx-quality-bench/outputs/cli/tune/tune-20260924-220158)

[Thomas-Kluge]

Follow-up: prefill is quantization-neutral on this machine too

Prompted by the question of whether the Quality pack also pays for its 8-bit weights during prefill (the part that hurts most on long agent contexts). The server's own request log (~/.mtplx/logs/request-log-1395.jsonl) happens to straddle the pack switch on this Mac: 314 cold prefills ≥1k tokens while serving Optimized Speed, then 11 while serving Optimized Quality, same machine, same daily Hermes/agent workload.

Median prefill_tok_s on cold prefills (new_prefill_tokens ≥ 1k):

Prompt size	Optimized Speed	Optimized Quality
1k–4k	597 tok/s (n=186)	830 tok/s (n=7)
4k–10k	843 tok/s (n=66)	963 tok/s (n=1)
30k–100k	849 tok/s (n=12)	908 tok/s (n=3)
largest single prompt	100,856 tok → 115.1 s = 876 tok/s	53,381 tok → 57.8 s = 923 tok/s

Readings:

• No prefill regression from the 8-bit pack. Saturated prefill sits at ~850–920 tok/s on the M3 Ultra in both eras, comfortably next to the published 810 tok/s for the Speed pack at 131k on the M5 Max. This matches expectation: prefill is FLOP-bound and both packs compute in bf16 after dequant, so the quantization only moves bytes over the bus, which is the decode-side story.
• Honest caveats: the Quality sample is small (n=11, only 3 requests in the 30k–100k bucket), and the Speed 1k–4k median is dragged down by many small cold requests shortly after boot. The data rules out "Quality is clearly slower on prefill"; it does not establish that Quality is faster.
• The one prefill-adjacent Quality cost that is real is per-boot: ~128.5 GiB of weights stream in from SSD instead of ~83 GiB, so the first request after a cold start arrives later. Once the engine is warm and the SSD session cache is hitting (2k–15k cached tokens per turn in these logs), the two packs behave the same.

Decode-side summary for this Mac stays as in the issue body: AR 39.9, D2 95.3 tok/s (2.39x), D2 saved as this machine's tuned depth.