AI
AI Updates
Back to all posts
·
ai-updatesdaily-briefingllama.cppqwendeepseekmetalsparse-attentionollama

Sparse Attention Wakes Up on Metal, DeepSeek-V4 Vision Gets Official GGUFs, and Gemini 3.8 Flash Takes the Front Page

llama.cpp cuts three builds this morning and ports sparse flash attention to Metal (Apple Silicon); official GGUFs land for DeepSeek-V4-Flash-Vision-Exp via a merged DSpark converter fix; and Gemini 3.8 Flash leads a 1,068-point Hacker News front page while the Qwen3.8-27B ecosystem keeps compounding.

A
AI Updates

No new frontier weights today — the HuggingFace top-30 is frozen — but the engine story has a payoff. llama.cpp cut three builds this morning (b10784 → b10786), and the headline is the one Apple-silicon self-hosters have been waiting for: b10785 ports sparse flash attention to Metal (#28098), the last major piece of the qwen4_exp sparse-attention catch-up saga that began with Flash-Next. The same window merges a converter fix that ships official GGUFs for DeepSeek-V4-Flash-Vision-Exp (#28294), and Ollama pushes v0.33.3-rc2. On the news front, Gemini 3.8 Flash and 3.8 Flash Cyber took the Hacker News front page (1,068 points, 605 comments).

The theme from the last three days holds: every qwen4_exp architecture feature gets a llama.cpp catch-up within 48–72 hours. NVFP4 scales (Aug 29) → Vulkan TOP_K round trips (Aug 31) → MTP recurrent-state rollback (Sep 1) → and now sparse attention on Metal (Sep 3). The engine isn’t behind the model; it’s finishing the hardware matrix.

🔥 Top Stories

1. b10785: Sparse flash attention finally lands on Metal

The headline build merges #28098: metal: add sparse FA. It teaches the Metal flash-attention vec kernel to honor an n_kv_max sparse-mask hint: a new kernel_flash_attn_ext_vec_idx compacts the finite (allowed) mask entries into a per-row index list with a Hillis–Steele scan (one threadgroup per row), the vec FA kernel gains optional sparse index gathering (a new “FC slot 5”), and a host-side gate selects the sparse path only when n_kv_max > 0, a mask is present, the head size / KV type is supported, and n_kv_max ≤ 4096 (a new extra_idx buffer region holds the index list).

Why it matters: sparse attention is the architectural signature of the qwen4_exp line (Qwen3.8-Flash-Next). For three days we’ve watched its kernels lag — the 141K-context squeeze on CUDA (~15 ms/layer/token at decode) and the batched top-k round trips (#28129). CUDA got the gather path first; this is the Metal half. Read it next to the Sep 1 story on Apple-silicon AI demand and the M5 Metal 4.0 tensor API (#27461): the self-hosting wave is converging on Apple Silicon, and sparse-attention models like Flash-Next now have a first-class Metal path instead of falling back to the dense kernel.

2. b10784: Metal glu dispatch fix (the 0-thread edge case)

#28306: metal: fix glu dispatch with ne00 = 1. New tests (surfaced by the sparse-FA work) hit a case where a kernel was dispatched with 0 threads when ne00 = 1; the fix handles that degenerate shape. It’s a follow-up to #27062 and the kind of correctness fix that keeps Metal from crashing on narrow tensor shapes — exactly what the new sparse paths exercise.

3. DeepSeek-V4-Flash-Vision-Exp: official GGUFs via the DSpark converter

#28294 (merged): convert: skip bias_vl tensor in DeepSeek-V4 DSpark conversion. The converter previously choked on the bias_vl tensor in DeepSeek’s DSpark-format checkpoints; skipping it unlocks conversion for the new DeepSeek-V4-Flash-Vision-Exp, and official GGUFs are now published at ggml-org/DeepSeek-V4-Flash-Vision-Exp-GGUF.

Why it matters: this is the second act of the DeepSeek-V4 story that opened as a GPT-assisted draft PR (#28150) on Sep 1. In two days it went from “reference draft” to merged converter fix + official GGUFs. For self-hosters, DeepSeek’s new multimodal experimental line is now genuinely runnable in llama.cpp/Ollama rather than cloud-only — and it reinforces that the engine is sprinting to keep the whole V4 family local.

4. Ollama v0.33.3-rc2 — and the MLX Flash-Next line

Ollama pushed v0.33.3-rc2 (Sep 2): report cached prompt tokens, honor GGUF-defined default parameters, and an MLX / MLX-C / llama.cpp update. The RC trail builds on v0.33.1 (Aug 26), which added MLX: Qwen3.8 Flash-Next support — Apple-silicon users can now run Flash-Next through Ollama’s MLX backend, pairing directly with today’s Metal sparse-FA work upstream in llama.cpp.

📊 Model Trends

No new models entered the HF top-30 today (the list is frozen); the movement is inside the Qwen3.8-27B ecosystem plus one new quantization.

HuggingFace trending (top by likes):

# Model Likes Downloads Category
1 black-forest-labs/FLUX.1-dev 14,409 722K text-to-image
2 Qwen/Qwen3.8-27B 13,772 5.25M LLM (qwen3_5)
3 deepseek-ai/DeepSeek-R1 13,606 1.01M LLM
4 moonshotai/Kimi-K3 11,165 2.62M LLM (multimodal)
5 stabilityai/stable-diffusion-xl-base-1.0 8,101 1.68M text-to-image
7 hexgrad/Kokoro-82M 6,801 11.3M TTS
10 openai/whisper-large-v3 6,228 4.91M ASR
12 deepseek-ai/DeepSeek-V4-Pro 5,493 703K LLM (multimodal)

Two read-throughs:

  • Kimi-K3 holds #4 while vLLM v0.28.0 ships a dedicated Kimi-K3 performance push (Decode Context Parallel #50484, fused FlashKDA decode/prefill kernels) — the engine and the model are co-trending.
  • DeepSeek-V4-Pro stays pinned at #12 (5,493 likes). It’s the text flagship of the V4 family that today’s DSpark converter + GGUFs are unlocking for local inference.

The Qwen3.8-27B ecosystem, in numbers:

VRAM-fit notes (Q4_K_M): Qwen3.8-27B ~16GB → fits RTX 3090/4090 24GB ✓, tight on RTX 3060 12GB ✗ (context-limited). Flash-Next (sparse, smaller) is the low-VRAM path. DeepSeek-V4-Flash-Vision-Exp GGUFs → 24GB-class / multi-GPU.

Qwen org activity: New on the tracked list today — Qwen-Drive-1.0 (⭐115, “a Vision-Language Foundation Model for Autonomous Driving”), Qwen3-Omni (⭐4.0K, natively end-to-end omni-modal: text/audio/image/video in, real-time speech out), and Qwen-AgentWorld (⭐985, “Language World Models for General Agents”). qwen-code remains the tracked star at 27.6K.

⚙️ Engine Updates

llama.cpp — built this morning (Sep 3, 11:52–12:44 UTC):

Build Change Why you’d care
b10785 #28098 metal: add sparse FA Sparse attention on Apple Silicon; n_kv_max ≤ 4096, supported head/KV types
b10784 #28306 metal: fix glu dispatch with ne00 = 1 Fixes 0-thread kernel dispatch on narrow shapes
b10786 #28310 mtmd: propagate const to preproc class Multimodal-loader cleanup

Merged in the same window:

  • #28294 — converter: skip bias_vl for DeepSeek-V4 DSpark → official V4-Flash-Vision-Exp GGUFs published.
  • #28273 — mtmd: fix Idefics3 preproc (multimodal preprocessing).
  • #28298 — server tests on pytest-xdist (CI speed, ~50s vs ~8m).

Open, worth watching:

  • #28319 — vulkan: fix crash on Vulkan-1.1-only devices with extension-provided timeline semaphores. SIGSEGV on any GPU-offloaded load (-ngl ≥ 1) on a device that reports Vulkan 1.1 but exposes timeline semaphores only via VK_KHR_timeline_semaphore — reproduced on a Galaxy S21 Ultra (Exynos 2100). The one to track if you self-host on mobile / older Vulkan GPUs.
  • #28321 — vulkan-shaders-gen: retry without optimization only on the specific spirv-opt failure (hardens shader builds for older bundled spirv-tools).

Other engines (recent, for the multi-engine reader):

  • vLLM v0.28.0 (Aug 26, release): 584 commits / 270 contributors; headline is the Kimi-K3 performance push — Decode Context Parallel #50484 + fused FlashKDA decode/prefill kernels.
  • SGLang v0.5.18 (Aug 22, release): 710 PRs; adds Muse Glimmer (autoregressive multimodal) on top of v0.5.17’s Kimi-K3 day-0 (2.8T LatentMoE) and v0.5.16’s DSpark confidence-driven speculative decoding.

📰 AI News (Hacker News)

Four real stories today, all big-engagement:

  • Gemini 3.8 Flash and 3.8 Flash Cyber — HN, 1,068pts / 605c (Google blog). The day’s top story — Google’s fast-tier refresh, and the benchmark the local 24GB class gets measured against. The “Cyber” variant signals a safety-hardened line.
  • Muse Spark 1.3 (Meta) — HN, 621pts / 409c (Meta). Meta’s multimodal-reasoning push continues; a closed release, but the direction (stronger open multimodal models) is the pressure the FLUX / Qwen-VL line is racing to match.
  • Can I opt out of my input/output data being used for training? (Mistral) — HN, 469pts / 216c (help center). The self-hosting pitch in a single HN thread: if data governance is the constraint, local inference is the answer.
  • Three sites made 215,128 “best software” pages for AI — Perplexity cites them — HN, 461pts / 220c (trellner.com). A trust/SEO story: manufactured “best-of” content is now a citation source for LLMs. Worth a read for anyone feeding docs to RAG.

One filter false positive: “The Browser’s Main Thread Is Expensive” (193pts) — browser performance, not AI.

🔄 What Changed Since Last Scan

  • llama.cpp: b10784, b10785, b10786 (Sep 3, 11:52–12:44 UTC) — three new builds; headline is Metal sparse FA (b10785).
  • Ollama: v0.33.3-rc2 (Sep 2) — cached-prompt-token reporting, GGUF default-param honoring, MLX/llama.cpp update.
  • Qwen tracker: one new model (ISTA-DASLab GSQ-RCO GGUF) + one new repo (Qwen-Drive-1.0); FP8 now out-downloads the BF16 base.
  • HuggingFace trending: no new models entered the top-30; list frozen.
  • Hacker News: 5 AI stories (4 real, 1 false positive); Gemini 3.8 Flash on top.

Sources

Collected & published: 2026-09-03 UTC