Model Intelligence — 2026-06-15

🔥 Top Stories

1. vLLM v0.23.0 — DeepSeek-V4 Gets Production Green Light

vLLM v0.23.0 dropped today with 408 commits from 200 contributors. The headline is the TRTLLM-gen attention kernel for DeepSeek-V4, making it genuinely production-ready.

Key additions:

• DeepSeek-V4 hardening: sparse MLA metadata decoupled from V3.2, EPLB support for Mega-MoE, XPU attention decode path, selective prefix-cache retention for sliding-window KV cache
• Model Runner V2 now default for Llama and Mistral dense models (plus Qwen3), with FlashInfer sampler and breakable CUDA graphs
• Rust frontend grew streaming generate, dynamic LoRA endpoints, request-ID headers
• Gemma 4 Unified encoder-free support + MTP
• Transformers v5 compatibility

Signal: The TRTLLM kernel is the missing piece for production DeepSeek-V4 serving. Between vLLM v0.23.0 and SGLang v0.5.13's full V4 inference path, the ecosystem now has two battle-tested options. Minimax M3 not yet supported — expect v0.24.

2. llama.cpp Hits b9660 — Chat & Tool Calling Hardened

llama.cpp reached b9660 today (from b9637 yesterday), spanning 13+ commits. Five builds landed today alone:

• b9655 — Grammar generator bug fix that surfaced during recent changes (#24653)
• b9656 — Hardened peg-native tool call parsing with optional leading type support (#24329)
• b9658 — Full unparsed prompt included in debug output (#24650)
• b9659 — Continued hardening
• b9660 — Latest build of the sprint

Also in the day's span: Metal bf16 repeat (Mac Studio M-series), SYCL Level Zero optimization (Intel GPUs), WebGPU i-quants performance, HEIC/HEIF image support, thinking/reasoning block rendering as markdown (#24611).

Signal: The tool call parsing hardening is quietly critical — if you're running function-calling models through llama.cpp, b9656+ is the update to grab. The reasoning-block markdown rendering matters for DeepSeek-R1 and chain-of-thought workflows.

3. Ollama v0.30.9-rc1 — Rolling Up to b9637

Ollama v0.30.9-rc1 landed today, updating the bundled llama.cpp to b9637. This means the Cohere2MoE parser and recent template fixes from yesterday's llama.cpp sprint are now in the Ollama pipeline.

Signal: RC releases typically ship within a week. If you're on Ollama for production, the v0.30.8→v0.30.9 upgrade brings 2+ days of llama.cpp fixes into a stable package.

📊 Model Trends

HuggingFace Top 15

Standouts: Kokoro-82M jumped +600K downloads to 11.7M — this is embedded-in-production velocity, not hobbyist testing. DeepSeek-V4-Pro climbing steadily at 4,857 likes. MiniLM-L6-v2 at 167M downloads remains the silent workhorse of embedding.

Qwen Family

Qwen3.6-35B-A3B remains the practical recommendation: MoE with 3B active params out of 35B total, runs on consumer hardware.

Gemma Family

gemma-4-31B-it at 7.5M downloads — Google's flagship with solid local inference support via Ollama's QAT weights.

⚙️ Engine Updates

Three engines shipped today — the busiest single day in recent weeks. If you haven't updated any inference stack in the last 48 hours, do it now.

📰 AI News (Hacker News)

• [514 pts] CrankGPT — crankgpt.com — New AI tool with massive HN engagement
• [449 pts] Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding? — HN — The question everyone's asking
• [100 pts] Can Europe train a frontier AI model on the compute it owns? — euromesh — European AI sovereignty exploration

The coding replacement thread (449 pts) is the most useful discussion — real practitioners sharing what works for local-first dev workflows.

🔄 What Changed Since Yesterday

Local Inference Recommendations

RTX 3060 (12GB):

• Qwen3.6-35B-A3B at Q4_K_M — MoE efficiency still unbeatable at 3B active params
• Gemma-4-31B-it at Q4 — 7.5M downloads validate the quality
• Kokoro-82M for TTS — production-grade, runs in milliseconds

RTX 3090 (24GB):

• DeepSeek-V4-Pro via vLLM v0.23.0 — TRTLLM kernel makes production serving practical now
• Qwen3.6-35B-A3B at Q6/Q8 — full quality with context headroom
• Gemma-4-31B-it at Q6 — high quality, extended conversations
• gpt-oss-120b at Q2_K — experimental but functional on 24GB

Key takeaway: Three engines shipping today makes this a must-update day. vLLM v0.23.0's TRTLLM kernel is the most actionable release — it unlocks production DeepSeek-V4 serving. llama.cpp's tool call parsing hardening (b9656) matters for function-calling workflows. Ollama's RC brings the day's llama.cpp fixes into a stable packaging pipeline.

Scan completed: 2026-06-15 | Sources: HuggingFace API, llama.cpp GitHub, Ollama GitHub, vLLM GitHub, SGLang GitHub, Hacker News