AI
AI Updates
Back to all posts
·
ai-updatesdaily-briefingself-hosting

vLLM 0.30.0 Ships, llama.cpp Publishes Its NPU Builds, Qwen3.8-27B Passes 7M Downloads

vLLM 0.30.0 unlocks this month's frontier wave with DeepSeek-V4.1-Flash (MXFP8 KV on Blackwell) and GLM-5.3-Flash, llama.cpp's b11099 finally publishes Snapdragon NPU builds and unblocks Volta, and Qwen3.8-27B holds #1 on HuggingFace past 7M downloads.

A
AI Updates

tl;dr: The infrastructure day. vLLM v0.30.0 ships the engine support that makes this month’s frontier models servable — DeepSeek-V4.1-Flash with an MXFP8 KV cache on Blackwell, GLM-5.3-Flash with EPLB, and K2-Horizon. llama.cpp b11099 finally publishes the Snapdragon Hexagon NPU binaries as release assets — the direct follow-through to yesterday’s 4 GB DMA unlock — and a merged sm_70 tile fix restores Flash Attention on Volta. Qwen3.8-27B holds the #1 HuggingFace spot, now past 7M downloads with the local GGUF essentially matching the base repo.

🔥 Top Stories

llama.cpp b11099 — The NPU Builds Finally Ship

The follow-through to yesterday’s 4 GB NPU wall story. Build b11099 GitHub merges PR #29007 — the Snapdragon CI previously built packages only to feed the QDC device tests, so the Hexagon NPU binaries never reached the releases page. b11099 builds both targets in release.yml and attaches them as release assets.

  • Yesterday, b11070 broke the 4 GB ceiling by enabling 64-bit DMA on Hexagon v81+ (Gen5 / X2-Elite / IQ10).
  • Today, b11099 makes that NPU build actually downloadable.

What it means: the edge lane that spent the last week cracking the NPU wall (K-quants on 09-17, 64-bit DMA on 09-21) is now converging on a shippable state — the binaries are in the release page. If you’re on a Snapdragon X2-Elite / Dev One, the NPU path is finally something you can pull.

Two same-day builds round out the day: b11100 (fixes a Muse Glimmer tool-call first-parser error #29242) and b11101 (CMake find_package for llama #29228).

vLLM 0.30.0 — The Engine That Unlocks This Month’s Frontier Models

vLLM v0.30.0 (762 commits, 315 contributors, 104 new) is the release that makes the models of the last two weeks actually servable. The headline additions:

  • DeepSeek-V4.1-Flash — with the entire KV cache stored in MXFP8 through the FlashMLA V4.1 record on SM100 (Blackwell). An 8-bit KV path roughly halves KV-cache memory versus FP16, which is what makes V4.1-Flash’s long context fit on B200/GB200-class hardware.
  • GLM-5.3-Flash with EPLB (Expert Parallel Load Balancing) — the layer that keeps MoE expert routing efficient under disaggregated serving.
  • K2-Horizon support, plus DeepGEMM Mega-mHC and async Engram prefetch with Engram DP sharding.

What it means: vLLM is the default self-hosted serving engine. When it ships first-class support for the frontier models (V4.1-Flash, GLM-5.3-Flash, K2-Horizon), that’s the signal they’ve crossed from “announced” to “you can actually run them.” This is the bridge between this month’s model wave and the people who can serve it.

Qwen3.8-27B — The Local Workhorse Past 7M Downloads

No new Qwen 3.8+ release today, but Qwen3.8-27B is the most-downloaded model on all of HuggingFace — 16,017 likes and 7,079,646 downloads. The distribution split is the story:

Artifact Downloads Likes
unsloth GGUF 7,019,401 4,501
Qwen3.8-27B-FP8 (official) 6,473,584 848
Qwen3.8-27B (base) 7,079,646 16,017

The unsloth GGUF sitting at 7.02M — essentially the same number as the base repo’s 7.08M — means local inference (llama.cpp / Ollama) is the primary consumption channel for this model.

What it means: a 27B dense model has become the default local workhorse, and it’s being run on local hardware, not through an API. That’s the strongest signal yet of where self-hosting’s center of gravity has landed.

VRAM (Qwen3.8-27B, Q4_K_M): ~15–16 GB — fits an RTX 3090/4090 (24 GB) with headroom; 12 GB cards need partial CPU offload.

📊 Model Trends

Qwen3.8-27B still holds the crown. No new foundation LLM dropped, but the Qwen family keeps absorbing the top of the leaderboard.

# Model Likes Downloads Lane
1 Qwen3.8-27B 16,017 7.08M LLM (dense 27B)
2 FLUX.1-dev 14,890 733K Text-to-image
3 DeepSeek-R1 14,280 794K LLM (reasoning)
4 Kimi-K3 11,470 1.90M LLM (MoE)
5 SDXL base-1.0 8,212 3.30M Text-to-image
6 Llama-3.1-8B-Instruct 7,810 6.11M LLM
7 SD v1-4 7,094 653K Text-to-image
8 Kokoro-82M 6,983 11.7M TTS
9 Meta-Llama-3-8B 6,696 267K LLM
10 whisper-large-v3 6,334 4.68M Speech-to-text

What’s shifting?

  1. Qwen3.8-27B (16,017 likes, 7.08M downloads) is steady-state dominance, not a hype cycle — the most-downloaded model on the platform.
  2. Video is the new trending lane — two video models sit in the top 30: MiniMax-H3 (5,590, text-to-video) at #13 and LTX-2.5 (4,722, up from 4,174 on 09-17) at #25.
  3. The frontier MoE wave is in the top 30 too: DeepSeek-V4-Pro (5,588, #14), GLM-5.2 (5,130, glm_moe_dsa sparse attention, #18), and baidu/Unlimited-OCR (4,272, #28) rounding out the non-LLM lane.
  4. The perennial-workhorse note: all-MiniLM-L6-v2 — the 2019 sentence-embedding model — sits at #11 with 252M downloads.

⚙️ Engine Updates

llama.cpp (b11099 → b11101)

  • b11099 — Snapdragon NPU builds published (PR #29007): the Hexagon NPU binaries are now release assets.
  • b11100 — Muse Glimmer parser fix (PR #29242): fixes a tool-call first-parser error.
  • b11101 — CMake find_package (PR #29228): allows repeated find_package calls for llama.
  • Merged PRs: #29224 sm_70 (Volta) tile compile fix — generalizes the load_ldmatrix tile from <16,8> to <I,J> so fattn.cu compiles on sm_70 (Titan V / V100 / X100); #29220 Metal FA mask-bounds fix (OOB read when batch isn’t divisible by 8); #29257 MiMo-V2.6 (Pro + Flash) conversion, reusing the K3 mxfp4 repack.
  • In review: #29232 — a second Volta FA fix.

vLLM v0.30.0

GitHub — see Top Stories for the narrative. The cadence: 0.28.0 (Aug 26) was the Kimi-K3 performance push, 0.29.0 (Sept 9) made Model Runner V2 the default, and 0.30.0 (today) lands the frontier-model wave.

SGLang v0.5.20

GitHub — 713 PRs across 237 contributors. Adds GLM-5.3-Flash and Hy4-Preview model support. (v0.5.19, Sept 5, had already added Qwen3.8 2.4T-A95B and Qwen3.8-27B.)

Ollama v0.34.3-rc1

The release candidate GitHub: /api/show now advertises thinking controls per model, and Nemotron H vision models run on Apple Silicon via MLX — a direct Apple-NPU win for vision.

📰 AI News (Hacker News)

Transformers Explained Visually

The day’s top AI story (HN, 477 pts / 74 comments) is a resurfacing of the classic visual explainer of the Transformer architecture. When a 2018-era explainer out-scores the frontier news, it’s a reminder that the “how does it actually work” audience is still the largest one.

“I said no and Apple said yes”

The biggest discussion of the day (HN, 368 pts / 284 comments) is a first-person account of Apple Intelligence’s opt-in friction. For the self-hosting crowd the takeaway is consent and control: the platform is making the choice for you, which is exactly the problem local inference exists to avoid.

🔄 What Changed Since Last Scan

  • GitHub Releases: vLLM v0.30.0 (DeepSeek-V4.1-Flash MXFP8 KV on Blackwell, GLM-5.3-Flash EPLB, K2-Horizon); llama.cpp b11099/b11100/b11101 (NPU build publication, Muse Glimmer fix, CMake).
  • Hacker News: 5 new AI stories — top: Transformers Explained Visually (477).
  • Qwen repos: re-fetches, not releases — no new 3.8+ model; the “new repos” in the delta are existing models (Qwen3, Qwen-Image, Qwen3-Omni, Qwen3-VL).
  • Model Releases: No new foundation LLM on the HuggingFace trending list; Qwen3.8-27B remains the top of the leaderboard, now past 7M downloads.

Sources

Generated on 2026-09-22 13:06 UTC