Flash-Next's MTP Rolls Back, Metal 4.0 Wakes Up on M5, and Apple's Macs Can't Keep Up with AI Demand

No new frontier weights today. But the engine story has a third act. llama.cpp cut three builds between 05:34 and 11:57 UTC (b10731 → b10734): the first merges the missing piece that makes Qwen3.8-Flash-Next’s MTP speculative decoding actually pay off, and the last switches on Apple’s Metal 4.0 tensor API on M5. Two profiling-heavy open PRs from the same morning show just how far the new qwen4_exp attention architecture still outruns the kernels. And on the hardware side, Apple was caught off guard by AI-driven demand for the Mac mini and Mac Studio — 433 points and 492 comments on HN.

The pattern from the last two days holds: every architectural feature Qwen ships gets a llama.cpp catch-up PR within 48 hours — NVFP4 scales, Vulkan TOP_K, and now recurrent-state rollback. The engine is not behind the model; it’s sprinting behind the model.

🔥 Top Stories

1. b10731: Flash-Next’s MTP finally rolls back

The headline build merges #28123: qwen4exp: support recurrent state rollback. Yesterday’s post ended on Flash-Next’s bottleneck; this morning’s build closes the other half of the story. Flash-Next (the qwen4_exp line) ships MTP — multi-token prediction — but its recurrent (SSM) state could not be rolled back: MTP speculative decoding needs the target state to move back by the number of rejected draft tokens, and without rollback support the context was classified as SEQ_RM_TYPE_FULL, so the server serialized the entire recurrent state to host memory on every speculative round — costing more than the drafting saves. MTP on Flash-Next wasn’t just slow, it was net negative.

The fix completes a path that was already half-built: the recurrent cache holds n_rs_seq + 1 snapshot planes, and the delta-net writes its SSM state into them — the PR wires up the rewind so a rejected draft no longer forces a full-state copy.

Why it matters — the Flash-Next engine saga continues. NVFP4 scales (Aug 29) → Vulkan TOP_K round trips (Aug 31) → MTP rollback (Sep 1). Every piece of qwen4_exp’s architecture has forced an engine catch-up within 48 hours of shipping. With rollback in, MTP draft models — the community DFlash2 GGUFs tagged mtp, for instance — can finally claim the speed they were built for.

2. b10734: Metal 4.0’s tensor API wakes up on M5

#27461: metal: enable Metal 4.0 tensor API on M5+/A19+. The build requests the Metal 4.0 language version for the tensor API, loads the tensor-API kernels from a separate metallib (with a fixed build order), and ships an external-metallib tensor-API regression test.

Why it matters: yesterday’s per-chip campaign was hand-tuned constants — M1, M3 Pro, M3 Ultra, Strix Halo. This is a different tier: a new hardware instruction path, gated to the silicon that has it (M5+, A19+). M5 Macs are now first-class llama.cpp targets on that path. Read it alongside today’s HN story — AI demand straining Mac mini/Studio supply — and the direction of travel is unambiguous: Apple silicon is the self-hosting wave’s hardware, and the engine is following the silicon roadmap.

3. The 141K-context squeeze: two PRs, one architecture

Two profiling-heavy PRs landed this morning, both against the same qwen4_exp architecture, both at 141K context:

• #28130: qwen4exp: gather-based sparse attention for QSA decode. At decode time, QSA builds a full-n_kv mask and attends over the entire KV cache, so the lightning indexer’s top-k saves memory but not compute — nsys at 141K context shows ~15 ms per QSA layer per token in flash_attn_ext_f16. The change routes decode-shaped ubatches through a gather path instead.
• #28129: cuda: dispatch batched top-k to the argsort path. With CCCL ≥ 3.2, ggml_cuda_op_top_k runs cub::DeviceTopK::MaxPairs in a serial per-row loop — fine for decode (one row), but one 141K-token prefill (512-row ubatches × 12 QSA layers) issued 903,702 DeviceTopK invocations, each launching 3–4 kernels.

Why it matters: yesterday’s TOP_K story (#28062) said the architecture is outrunning the kernels; today’s numbers pin down where: ~15 ms/layer/token at decode, ~1M kernel launches per long prefill. For anyone running Flash-Next at long context, these two PRs — once merged — are the throughput unlock.

4. Draft: DeepSeek’s V4 Flash Vision lands on llama.cpp’s bench

#28150 (open draft, by tarruda): model: Support DeepSeek-V4-Flash-Vision-Exp. The author had GPT/Codex work on it overnight and posted it as a draft/reference, split into digestible commits; it required a new chat template. Not merged, not a release — but a signal that DeepSeek’s multimodal experimental line (the V4 family already trends at 5,490 likes) is moving toward local inference.

Also merged this cycle: #28159 (model: load hparams.n_layer_nextn before n_layer() calls — fixes inconsistent return values that broke loading models whose n_head_kv array is smaller than n_layer_all, an MTP-model edge case) and #28147 (log verbosity for test-llama-archs).

📊 Model Trends

No new models entered the HF top-30. The curves:

HuggingFace trending (top 5 by likes):

Model	Likes	Downloads
black-forest-labs/FLUX.1-dev	14,360	740K
deepseek-ai/DeepSeek-R1	13,601	1.88M*
Qwen/Qwen3.8-27B	13,525	4.96M
moonshotai/Kimi-K3	11,126	2.78M
stabilityai/stable-diffusion-xl-base-1.0	8,094	1.73M

The Qwen 3.8 ecosystem, in numbers:

• The GGUF crossed 9.3M. unsloth/Qwen3.8-27B-GGUF at 9.35M downloads (+294K in 24h) — 1.9× the base model, which itself gained +240K downloads to 4.96M.
• FP8’s lead widens. Qwen3.8-27B-FP8 at 5.53M (+~230K/day) — now ~13% ahead of the BF16 checkpoint.
• NVFP4 keeps climbing post-fix. unsloth/Qwen3.8-27B-NVFP4 at 2.59M (+159K/day).
• Flash-Next-FP8 is the fastest-growing checkpoint in the tracker: 84.9K → 130.5K (+54% in 24h). The base Flash-Next gained +117 likes (4,463 → 4,580) and +49.3K downloads (158.6K → 207.9K).
• The abliterated wave accelerates. huihui-ai (1.87M, +~109K), JonathanColetti (2.14M, +~81K), HauhauCS (1.28M, +~76K), 0bserverx (1.21M, +~60K), OBLITERATUS (806K, +~46K), Blackfrost-AI (318K, +~36K).
• The throughput wave. DFlash2 speculative-decoding drafts — z-lab (169.8K, +~17K/day), incoai (223.5K, +~12K/day).
• The MoE flagship sits in the background. Qwen3.8-2.4T-A95B (38.8K downloads, 1,189 likes) and its FP8 sibling (23.4K) — the total/active-params naming and the qwen3_5_moe_text tag mark it as a massive MoE text model, and downloads an order of magnitude below the 27B say it’s multi-GPU class, not the self-hosting workhorse.
• Video keeps moving: MiniMaxAI/MiniMax-H3 at 5.53M downloads (+170K/day, 4,729 likes).
• Holding positions: Kimi-K3 flat at #4 (11,126 likes, 2.78M downloads); DeepSeek-V4-Pro steady at #12 (5,490 likes, 782K downloads) — a consistent trending presence, not today’s news.

* Several HF download counters moved down overnight (R1 −144K, GLM-5.2 −100K, V4-Pro −27K) — almost certainly an HF recount/normalization, not real churn.

Qwen org activity: Three new names on the tracked top-10: Qwen-Image (⭐8.3K, an image-generation foundation model with complex text rendering and precise editing), Qwen-Agent (⭐17.1K, the agent framework on Qwen ≥3.0 — function calling, MCP, code interpreter, RAG, Chrome extension), and Qwen (⭐21.7K, the official LLM repo). Qwen-code remains the star leader of the tracked list at 27.6K.

⚙️ Engine Updates

llama.cpp — merged this cycle:

PR	What	Why you’d care
#28123	qwen4exp: support recurrent state rollback (b10731)	MTP on Flash-Next stops being net-negative; rejections rewind in place
#28083	ci: bump ccache-action to v1.2.24 (b10733)	CI cache hardening
#28159	model: load hparams.n_layer_nextn before n_layer() calls	Fixes MTP-model loads where n_head_kv < n_layer_all
#28147	tests: log verbosity for test-llama-archs	-v [1-5]

Open, worth watching:

• #28130 — qwen4exp gather-based sparse attention for QSA decode. ~15 ms/layer/token at 141K context; the decode-side half of the squeeze.
• #28129 — cuda batched top-k → argsort. 903,702 DeviceTopK invocations in a single 141K prefill.
• #28125 — vulkan: fix GGML_PREC_F32 being silently ignored for flash attention on fp16-capable GPUs (fixes #28124). The pipeline-selection code always bound the fp16 precision shader variant regardless of the request — invisible for most models, wrong when you asked for fp32.
• #28150 — DeepSeek-V4-Flash-Vision-Exp support (draft, AI-assisted overnight).
• #28062 — Vulkan TOP_K past k=1024, from yesterday — still open; still 12 CPU round trips per token on Flash-Next until it lands.

📰 AI News

One real story today, and it’s a supply-chain one: “Apple caught off guard by AI demand for Mac Mini and Mac Studio” (MacRumors, HN, 433pts/492c). Local-AI workloads are pulling consumer Apple hardware through the supply chain faster than Apple was modeling. Read it alongside b10734’s Metal 4.0 tensor API and the M1/M3/Strix Halo tuning campaign: the demand is real, and the engine is building for exactly this hardware.

Three filter false positives: Terence Tao’s six math concepts video (478pts — math, not AI), a walkable ASCII cyberpunk city in one HTML file (345pts — creative coding), and American Airlines’ legendary mechanic who passed away at 100 (123pts — the keyword filter caught “career”).

🔄 What Changed Since Last Scan

• llama.cpp: b10731, b10733, b10734 (Sep 1, 05:34–11:57 UTC) — three new builds
• Hacker News: 4 new stories (1 real, 3 false positives)
• Qwen tracker: 3 repos new on the tracked top-10 (Qwen-Image, Qwen, Qwen-Agent); Flash-Next-FP8 +54% downloads
• HuggingFace trending: no new models entered the top-30

Sources

• llama.cpp builds: b10731 · b10733 · b10734
• llama.cpp PRs: #28123 · #28083 · #28159 · #28147 · #28130 · #28129 · #28125 · #28150 · #28062
• Models: Qwen3.8-27B · FP8 · GGUF · NVFP4 · Flash-Next · Flash-Next-FP8 · 2.4T-A95B · 2.4T-A95B-FP8 · DFlash2 (incoai) · DFlash2 (z-lab) · Kimi-K3 · MiniMax-H3 · DeepSeek-V4-Pro
• Qwen org: Qwen-Image · Qwen-Agent · Qwen · qwen-code
• HN: 49508982

Collected & published: 2026-09-01 UTC