AI
AI Updates
Back to all posts
·
ai-updatesdaily-briefingllama.cppollamaqwenapple-siliconrocm

llama.cpp Tunes M3 Max & M5 Metal, Adds a Qwen3.8 MTP Draft Head, Ollama v0.33.2 RC

A no-new-model day that's all engine: llama.cpp lands fa-vec Metal tunings for M4 Pro, M4, and M3 Max/M5/M5 Pro across b10667–b10669, opens the MTP-draft-head and ROCm hipCUB path for Qwen3.8-Flash-Next, and Ollama ships v0.33.2-rc1. HF trending is flat; Google drops two Gemini omni/Transcribe models that trend on HN.

A
AI Updates

Today is a no-new-model day, and the movement is all in the engines: Hugging Face trending is flat — FLUX.1-dev still holds #1 with no new release to displace it — so the signal is in the runtimes. llama.cpp shipped a three-build Apple Silicon Metal tuning wave (b10667–b10669) and opened the path for Qwen3.8-Flash-Next to MTP-based speculative decoding and a ROCm fix, while Ollama moved the accelerated cadence forward with v0.33.2-rc1; the only closed-source counterpoint trending on Hacker News is Google’s two new omni-modal Gemini drops (Transcribe and Omni 1.1 Flash).

Top Stories

1. llama.cpp b10667–b10669: The Apple Silicon Metal Tuning Wave

Three consecutive builds on 2026-08-28 (11:53 → 12:44 UTC):

  • b10667 — metal: add fa-vec tunings for M4 Pro (PR #27824, merged) — release
  • b10668 — metal: add fa-vec tunings for M3 Max, M5 and M5 Pro (PR #27863, merged) — release
  • b10669 — sycl: bind the f16 KV cache in place for the oneDNN SDPA path (PR #27468, merged) — release

What actually shipped: per-chip flash-attention (fa-vec) tuning for the newest Apple GPUs — M4 Pro, M4, M3 Max, M5, and M5 Pro — a follow-up to the community Metal-tuning effort in discussion #27668. (M4 itself is PR #27875, also merged.) b10669’s SYCL/oneDNN change binds the f16 KV cache in place, cutting KV memory traffic — the release notes cite ~4.56 GB/ubatch at a 34,816-token KV length on Qwen3.8-27B Q4_K_S.

Why it matters: M3/M4/M5 self-hosters get faster Metal attention with no config change; the SYCL change is an Intel/oneDNN efficiency win.

2. Qwen3.8-Flash-Next: The Engine Path Matures

Qwen3.8-Flash-Next is the day’s riser: 4,064 likes on 8/28 (up from 3,847 on 8/27, +217), still just off the top 10. Two open llama.cpp PRs move it forward — neither is in a nightly yet:

What actually shipped: nothing in a nightly yet — the day’s movement is two open llama.cpp PRs:

  • PR #27842 (open) — model: add MTP (nextn) speculative head for Qwen3.8-Flash-Next (qwen4exp). The official checkpoint ships 31 mtp.* tensors the converter currently drops, so the model’s own draft head is unavailable (only ngram-mod speculation works today); this wires the MTP draft head end-to-end for real speculative decoding.
  • PR #27874 (open) — hip: enable GGML_CUDA_USE_CUB via hipCUB (fixes TOP_K CPU-fallback cliff / RPC abort on ROCm). Fixes a severe ROCm decode cliff for Flash-Next: decode collapses from 19.6 → ~6 t/s once n_kv crosses ~1024 tokens (issue #27856).

Why it matters: Ollama’s MLX path for Flash-Next already shipped in v0.33.1 yesterday; today’s open PRs extend that to MTP-based speculation (all backends) and fix ROCm. Status is in flight, not shipped — do not describe MTP or the ROCm fix as available.

3. Ollama v0.33.2-rc1

Ollama v0.33.2-rc1 — the next RC in the accelerated cadence (v0.33.0 → v0.33.1 stable → this), published the evening of Aug 27 and surfaced in today’s scan.

What actually shipped: restore system dark mode (PR #18049), continue proxy requests when the model catalog changes (PR #18058), and synchronize the macOS app handoff (PR #18056).

Why it matters: it’s an RC, not stable — the cadence pattern (RCs clustering → stable) is what to watch.

4. Housekeeping

  • PR #27843 (merged) — OpenVINO bumped to 2026.3.1 with whisper.cpp support, Qwen3.5 on NPU, and new ops (Intel NPU path).
  • PR #27816 (merged) — spec: add DFlash2 support (local convolution + candidate selector).
  • PR #27811 (open) — a model-discovery dialog for HuggingFace GGUF browsing in the UI.

Trending is flat — no new model. Top 10 by likes (likes / downloads):

Model Likes Downloads
FLUX.1-dev 14,275 647,530
DeepSeek-R1 13,589 3,297,341
Qwen3.8-27B 13,084 3,457,687
Kimi-K3 11,050 2,675,145
SDXL-base-1.0 8,076 1,521,068
SD-v1-4 7,055 459,440
Kokoro-82M 6,762 11,273,585
Llama-3.1-8B-Instruct 6,685 5,869,115
Meta-Llama-3-8B 6,633 931,239
whisper-large-v3 6,191 4,359,272

Three data points worth a look:

  • Trending is flat: FLUX.1-dev holds #1 (14,275, +10 vs 8/27); Qwen3.8-27B is the only real mover (+111 → 13,084, #3).
  • Qwen3.8-27B-FP8 (3,974,725 dl) still outs the base (3,457,687) — the quant-first / datacenter pull-and-quantize pattern continues.
  • Just off the list: Qwen3.8-Flash-Next (4,064 likes, still climbing) and all-MiniLM-L6-v2, the ~250M-download embedding workhorse.

AI News

  • The load-bearing vocabulary of Claude — 591 points, 285 comments (the day’s biggest AI HN story). Article · HN. Why it matters: an empirical look at which words actually steer model behavior.
  • Gemini-3.5-Transcribe — 320 points. Google · HN
  • Gemini Omni 1.1 Flash — 280 points. Google · HN. Note both as Google’s “omni-modal” (voice+vision+text) direction.
  • Terminal-Bench-Science — evaluating AI agents on scientific research workflows, 104 points. Announcement · HN
  • Luanti removed from Google Play over a baseless AI copyright notice — 94 points. Blog · HN

What Changed Since Last Scan

Delta vs 2026-08-27 — 4 new releases, 6 Qwen org repo updates, 8 new HN stories, and no new HF models:

  • New llama.cpp releases: b10667, b10668, b10669 (Metal fa-vec M4 Pro / M3 Max-M5-M5 Pro; SYCL oneDNN f16 KV cache)
  • New Ollama release: v0.33.2-rc1
  • New HF models: none (trending flat)
  • Notable open llama.cpp PRs: #27842 (Flash-Next MTP draft head), #27874 (ROCm hipCUB top-k), #27811 (HF GGUF model-discovery UI)
  • Merged llama.cpp PRs: #27843 (OpenVINO 2026.3.1 + whisper.cpp + Qwen3.5 NPU), #27816 (DFlash2), #27824 / #27863 / #27875 (Metal fa-vec)
  • Qwen org repos updated (8/28): Qwen-RobotNav (new, 160★), Qwen3-Coder (16.8k★), Qwen-Image (8.3k★), Qwen3-TTS (13.1k★), Qwen-Image-Layered (2.1k★), Qwen3-ASR (3.4k★) — QwenLM org
  • New HN stories: 8

Summary: a no-new-model day that moved entirely in the engines — llama.cpp shipped the Apple Silicon Metal fa-vec tuning wave (M4 Pro, M4, M3 Max, M5/M5 Pro) plus the SYCL oneDNN KV-cache win, opened the Flash-Next MTP and ROCm paths, and Ollama pushed the cadence forward with v0.33.2-rc1, while Google’s two omni-modal Gemini drops were the closed-source counterpoint on HN.

Sources

Last updated: 2026-08-28