Monday, September 7, 2026·

Qwen3.8 Surges to #2, llama.cpp Gains RDNA4 & Apple Metal Support

Qwen3.8-27B challenges the Llama duopoly while llama.cpp b10839 delivers critical RDNA4 and M5 Max optimizations for self-hosters.

AI Updates

🔥 Top Stories

Qwen3.8-27B: The Challenger to the Throne

The Qwen ecosystem is heating up with the official release of Qwen3.8-27B. With over 14,000 likes on HuggingFace, it’s already overtaking older Llama 3.1 variants in the community trending list. This isn’t just a minor patch; the naming convention shift from 2.5 to “3.8” implies a significant architecture overhaul.

For the self-hosting crowd, the 27B parameter count is the sweet spot between capability and hardware requirements.

VRAM: Q4 quantized, this model requires roughly 15GB of VRAM for comfortable inference with context.
Hardware: Fits RTX 3090/4090 (24GB) ✓. Needs 8+ RTX 3090s for 3060 (12GB) ✗.
Community: Already seeing active forks like unsloth/Qwen3.8-27B-GGUF and JonathanColetti/Qwen3.8-27B-Uncensored-GGUF, signaling high demand for local deployment.

DeepSeek-V4-Pro Enters the Chat

DeepSeek is making a move. DeepSeek-V4-Pro has surged to the top 15 of the HF trending list with 5,500+ likes. While DeepSeek-R1 is the reasoning giant, V4-Pro is likely optimized for speed and coding tasks. This adds a new layer of competition to the 2026 “Big Three” (Llama, Qwen, DeepSeek), forcing self-hosters to consider more models for specialized tasks.

FLUX.1-dev Holds Image Gen Dominance

In the generative space, black-forest-labs/FLUX.1-dev remains the undisputed king with 14,500 likes. It continues to outperform older models like Stable Diffusion XL in terms of adherence and photorealism.

VRAM: 12B parameters in Q4 quantization require roughly 7GB of VRAM.
Hardware: Fits RTX 3060 12GB ✓. Even lower-end cards can manage this.
Implication: If you’re running local image generation, FLUX is the baseline. The “schnell” variant is still relevant for speed, but dev is the quality choice.

📊 Model Trends

The landscape is shifting away from single-model dominance toward specialized families. While Llama 3.1 remains a staple, Qwen’s 3.8 series is gaining massive traction.

Model	Likes	Category	VRAM (Q4)	Hardware Fit
Qwen3.8-27B	14,194	LLM	~15GB	RTX 3090/4090 ✓
FLUX.1-dev	14,501	Image Gen	~7GB	RTX 3060 12GB ✓
DeepSeek-R1	13,617	Reasoning	~16GB (Base)	RTX 3090/4090 ✓
Kimi-K3	11,223	Multimodal	~10GB	RTX 3060 12GB ✓
DeepSeek-V4-Pro	5,508	LLM	~12GB	RTX 3060 12GB ✓

What’s shifting? The rise of Kimi-K3 suggests a shift toward long-context and multimodal capabilities on consumer hardware, moving away from pure text-only models. The download counts for Qwen3.8 (6.7M downloads for FP8 variant) indicate that compression is becoming a priority as VRAM costs rise.

⚙️ Engine Updates

This is where the rubber meets the road for self-hosters. We’ve got critical hardware support updates that make non-NVIDIA GPUs viable for production inference.

llama.cpp b10839: RDNA4 & Metal Support

The latest build of llama.cpp (b10839) is a game-changer for AMD and Apple users.

RDNA4 Support: PR #28529 enables WMMA flash attention for head size 256 on RDNA4. This means AMD’s newest GPUs (RDNA4 architecture) will finally see the performance boost previously locked to NVIDIA CUDA.
Apple Metal: PR #28534 optimizes M5 Max D512 decode in the existing KV bucket. For Mac Pro users with M5 chips, inference should be significantly faster.
Stability: Fixes races in mmid and mmf (PR #28475) ensure more stable long-context runs.
Falls back gracefully: PR #28465 ensures the model falls back to CPU when a KV tensor is too large for the device, preventing crashes on mixed setups.

vLLM & SGLang: The AMD Speculative Decoding Breakthrough

While vLLM v0.28.0 and SGLang v0.5.19 are the latest stable releases, the real news is the community discussion around Speculative Decoding on AMD GPUs.

Impact: HN Post highlights that AMD GPUs can now participate in speculative decoding, not just NVIDIA. This breaks the “NVIDIA-only” speed advantage.
Self-Host Takeaway: If you have an RX 7900 XTX or newer RDNA4 cards, you can now run LLMs at speeds approaching NVIDIA A100s for similar workloads. This changes the cost-per-token calculation for self-hosted clusters.

Ollama 0.34.0-rc1

The rolling release of Ollama continues to mature. Version 0.34.0-rc1 (released 2026-09-05) integrates these backend improvements from llama.cpp and vLLM. Keep your local instances updated to leverage the RDNA4 optimizations.

📰 AI News

vLLM on AMD: Breaking the NVIDIA Bottleneck

A high-scoring HN thread discusses Speculative Decoding in vLLM on AMD GPUs. This is critical for anyone running inference on non-NVIDIA hardware. Previously, AMD RDNA cards were limited in throughput for large models. The new speculative decoding support allows for faster token generation rates, making AMD GPUs a genuine competitor in the inference market.

Hardware Longevity Concerns

Another trending story on HN (68 points) discusses smartphone makers not complying with EU repairability requirements. While not directly about models, this highlights the broader hardware ecosystem context. If you are building a dedicated AI server, consider the longevity of components. The trend toward proprietary repairability suggests that investing in modular, open-source hardware (like used enterprise GPUs) is becoming more important than consumer-grade laptops.

🔄 What Changed Since Last Scan

Since our last collection on 2026-09-06, three key GitHub releases have dropped, specifically targeting inference performance:

ggerganov/llama.cpp b10839 (Today): Adds RDNA4 WMMA flash attention.
ggerganov/llama.cpp b10837 (Today): Metal optimizations for M5 Max.
ggerganov/llama.cpp b10835 (Today): General stability and URL parsing fixes.
ollama/ollama v0.34.0-rc1: Updated to support new backend engines.
sgl-project/sglang v0.5.19: Latest release with improved throughput.

No new models were released this week, but the existing trending list (Qwen3.8, FLUX.1-dev) has gained significant traction, indicating strong organic growth rather than hype cycles.