Model Intelligence — 2026-06-19

🔥 Top Stories

1. llama.cpp Goes Hyperactive — 23 Builds in 24 Hours

If you run local inference, today is a big day for llama.cpp. The project shipped 23 new builds between b9704 and b9727, the most aggressive development sprint we've tracked in weeks. Here's what matters for practitioners:

• Eagle3 speculative decoding support for Qwen3.5 & 3.6 (b9723): This is the headline feature. Eagle3 is a speculative decoding strategy that can dramatically accelerate inference on models where it's applicable. Qwen3.5 and Qwen3.6 users on llama.cpp should see meaningful throughput gains once GGUFs with Eagle3 checkpoints become available.

• MTMD (Multimodal) overhaul (b9711, b9713, b9716, b9724): Four separate MTMD commits today covering InternVL batching, LLaVA-UHD image handling refactors, CLI batching, video test suites, and several bug fixes. If you use multimodal models through llama.cpp, this is a stabilization cycle.

• Power10 Q8/Q4 MMA matmul (b9717): IBM POWER10 users get optimized matrix multiplication using the machine's MMA (Matrix-Multiply-Accumulate) instructions for Q8 and Q4 quantization formats. Niche but impactful for enterprise Power systems.

• Server schema validation (b9707) and context shifting fix (b9722): Infrastructure hardening for the llama.cpp HTTP server. The n_discard fix for context shifting is especially important for long-context workloads where sliding windows need to stay consistent.

Verdict: Update to b9727 if you use Qwen3.5/3.6 or multimodal models. The Eagle3 support alone justifies the upgrade for anyone running those models.

2. Noam Shazeer Joins OpenAI

Noam Shazeer — co-creator of the Transformer architecture (the paper that underpins virtually every major model in this report) and former Google Brain lead — has joined OpenAI. This is a significant personnel move with implications for the competitive landscape:

• Transformer expertise at OpenAI: Shazeer's deep knowledge of model architecture design brings theoretical depth to a team that's been pushing the frontier of scaling and capability.
• OpenAI's talent war: The move signals OpenAI's continued aggressive hiring, potentially drawing top researchers from competitors.
• For local inference users: The indirect effect is what matters — when leading researchers join companies, their work shapes what gets open-sourced (or doesn't). Watch for what OpenAI's open-weight models look like under Shazeer's influence.

Source: Twitter: NoamShazeer | HN score: 344

3. vLLM 0.23.0 — The "Enterprise" Release

Released June 15 but still worth tracking: vLLM 0.23.0 is the largest release we've seen from the team — 408 commits from 200 contributors. Key features for local inference practitioners:

• DeepSeek-V4 hardening: If you're serving DeepSeek models at scale, this release has production-grade optimizations.
• Multi-Query Routing (MRv2) for Llama/Mistral: Improved attention patterns that reduce KV cache pressure — critical for fitting larger context windows on constrained VRAM.
• Gemma 4 Unified support: Google's latest Gemma models work with vLLM's unified pipeline.
• Multi-tier KV cache: The ability to spill KV cache across tiers (HBM → system RAM → SSD) means you can run larger effective context windows than your VRAM would suggest.

RTX 3060 (12GB) relevance: Multi-tier KV cache lets you run 30B-class models with longer context by spilling to system RAM. Slow, but functional for batch processing.

RTX 3090 (24GB) relevance: Full benefit from MRv2 — 30B-parameter models with 8K+ context windows now fit comfortably with Q4_K_M quantization.

📊 Model Trends

HuggingFace Trending (Top 15 by Likes)

Analysis: DeepSeek-R1 continues its reign at #1, but the gap with FLUX.1-dev is closing (just ~142 likes). The DeepSeek-V4-Pro is gaining traction at +23 likes/day. Notable: openai/gpt-oss-120b holds at #14 — the 120B open-weight model that's become a benchmark for "can it run locally?" discussions.

VRAM fit guide:

• RTX 3060 (12GB): Llama-3-8B (Q4: ~5GB, Q8: ~9GB), Kokoro-82M (trivial), Whisper-large-v3 (Q4: ~3GB)
• RTX 3090 (24GB): DeepSeek-V4-Pro (Q4: ~18GB), gpt-oss-120b (Q2: ~20GB — marginal), Llama-3-8B (Q8: ~16GB, FP16: ~16GB)

Qwen Model Tracker

Key observation: Qwen3.6-35B-A3B is gaining momentum, jumping to #5 in the Qwen rankings. The uncensored variant is growing at 33 likes/day — strong community demand for unfiltered versions. The Eagle3 spec support in today's llama.cpp release directly benefits Qwen3.5 and Qwen3.6 models.

VRAM fit guide:

• RTX 3060 (12GB): Qwen3.6-27B (Q4: ~15GB — won't fit), Qwen2.5-Omni-7B (Q4: ~5GB ✅, Q8: ~9GB ✅), QwQ-32B (Q4: ~17GB — won't fit)
• RTX 3090 (24GB): QwQ-32B (Q4: ~17GB ✅), Qwen3.6-35B-A3B (Q4: ~19GB ✅), Qwen3.5-27B (Q4: ~15GB ✅, Q5: ~17GB ✅)

Gemma Model Tracker

Key observation: Gemma-4-31B-it continues as the most popular G4 model, gaining 12 likes/day. The G4 family (31B and E4B) is showing steady growth, suggesting Google's latest generation is finding its footing. The MoE variant (26B-A4B) is still below 1.2K but growing consistently.

VRAM fit guide:

• RTX 3060 (12GB): gemma-7b (Q4: ~4GB ✅, Q8: ~8GB ✅), gemma-3-4b-it (Q8: ~5GB ✅), gemma-2-2b-it (Q8: ~2GB ✅)
• RTX 3090 (24GB): gemma-4-31B-it (Q4: ~17GB ✅), gemma-3-27b-it (Q4: ~15GB ✅, Q5: ~17GB ✅)

⚙️ Engine Updates

llama.cpp — 23 New Builds (b9704 → b9727)

Today was a landmark development day. The full changelog for today's builds:

Priority update: b9723 (Eagle3 for Qwen3.5/3.6) and b9722 (context shifting fix) are the most impactful for local inference users.

Ollama — v0.30.10 (June 17)

No new release today. Last release: v0.30.10 with Cohere2MoE support. Stable cycle.

vLLM — v0.23.0 (June 15)

No new release today. v0.23.0 remains current with DeepSeek-V4 hardening, MRv2, and multi-tier KV cache. The Rust frontend addition is a long-term architectural shift worth watching.

SGLang — v0.5.13 (June 13)

No new release today. v0.5.13 is current.

📰 AI News (Hacker News)

Today's top AI-related stories on Hacker News:

1. [416] Show HN: Are You in the Weights? — A new project exploring what data is embedded in model weights. Fascinating for anyone curious about training data provenance and reproducibility. Link

2. [344] Noam Shazeer Joins OpenAI — The Transformer co-author's move to OpenAI is generating significant discussion. What does this mean for open-weight research? Link

3. [164] Zen and the Art of Machine Learning Research — A reflective piece on ML research methodology. Useful reading for anyone building custom local models or fine-tuning. Link

🔄 What Changed Since Yesterday

Significant Changes

• llama.cpp: +23 builds (b9704 → b9727). Eagle3 spec decoding for Qwen3.5/3.6 is the headline. MTMD stability pass with 5 separate commits.
• DeepSeek-V4-Pro: +23 likes (fastest growing trending model, now at 4,961)
• Qwen3.6-35B-A3B: +12 likes, climbed to #5 in Qwen rankings
• HauhauCS/Qwen3.6-35B-A3B-Uncensored: +33 likes (fastest growing Qwen variant)

Minor Changes (Expected Daily Drift)

• All top-15 HF models gained 1-15 likes each
• gemma-4-31B-it: +12 likes (steady G4 adoption)
• Ollama, vLLM, SGLang: no new releases

What's Stable

• DeepSeek-R1 remains #1 on HF with 13,400 likes
• FLUX.1-dev at #2, closing the gap
• Llama-3-8B family unchanged in position
• Kokoro-82M continues its TTS dominance

Generated by: Hermes Agent Model Intelligence Tracker Sources: HuggingFace API, llama.cpp releases, Ollama releases, vLLM releases, SGLang releases, Hacker News