AI
AI Updates
Back to all posts
·
qwen3.8benchmarkslocal-llmai-updatesdeep-dive

Qwen3.8-27B: The 27B That Out-Drew Its Own 2.4T Flagship

The benchmarks, real-world tok/s, and community reception behind the 27B model pulling 10x the likes of Qwen's own 2.4T flagship — and what the speed numbers actually say. A deep dive.

M
Model Intelligence

Qwen3.8-27B: The 27B That Out-Drew Its Own 2.4T Flagship

Today’s daily briefing covered the logistics of a strange inversion: the community quantized Qwen3.8-27B before the official card even existed, and then the official weights arrived to a crowd that was already seated. This post is about the numbers underneath the frenzy — what the model actually is, what Qwen’s official benchmarks say, what users are actually getting on real hardware, and why a dense 27B just out-drew its own 2.4-trillion-parameter flagship by an order of magnitude.

What It Actually Is

Strip away the like-count drama and Qwen3.8-27B is a deliberately deployment-shaped model. Per the official card, it’s a dense native vision-language model in the qwen3_5 architecture family (Qwen3_5ForConditionalGeneration), tagged image-text-to-text with multi-modal input across text, image, and video. Dense matters: the contrast piece of this family is the Qwen3.8-2.4T-A95B, a text-only MoE with 512 experts (10 active per token) that needs a datacenter to serve. The 27B is the model you can put on a machine you own.

The card’s other notable specs:

  • 1M-token context advertised, with the recommended settings noting 262,144 tokens natively supported and YaRN recommended for extension — so treat “1M” as the marketed ceiling and 256K as the comfortable native range.
  • Thinking mode on by default, with adjustable reasoning effort (xhigh default, medium, low) — the model emits think blocks unless you turn it off.
  • Apache 2.0 — clean commercial terms, versus the flagship’s other license. For a lot of teams, that license delta alone is the story.
  • Recommended sampling: temperature=1.0, top_p=0.95 in thinking mode; temperature=0.7, top_p=0.80 for instruct.

The Official Numbers

Qwen’s card publishes a wide benchmark spread. The rows that matter most for the local-AI audience:

Benchmark Qwen3.8-27B What it measures
MMLU 88.3 Broad knowledge/understanding
MATH 91.3 Math problem solving
GPQA 62.4 Graduate-level science reasoning
LiveCodeBench 74.0 Advanced coding (contamination-resistant)
HumanEval 90.2 Code generation
SWE-bench Pro 61.7 Real-world software engineering
OSWorld-Verified 84.3 Computer use
WebArena-Verified 64.8 Browser use
AndroidWorld 81.9 Mobile use
MathVision 94.6 Visual math
OmniDocBench 1.5 91.4 Document understanding

All figures from the official model card. The agentic cluster is the interesting part: 84.3 on OSWorld-Verified and 81.9 on AndroidWorld for a 27B is the kind of computer-use score that usually belongs to models several times its size.

The generational jump: against the prior-gen Qwen3.6-27B, the official card shows SWE-bench Pro up from 53.5 to 61.7 and CoWorkBench up from 61.0 to 70.7. That’s not a patch — it’s a step.

The big claim: the card also asserts wins over Opus 4.6 Max on most agentic benchmarks, including SWE-bench 61.7 vs 53.4. That is Qwen’s official claim, not an independent measurement — vendor-published head-to-heads against a competitor’s flagship deserve a discount until a third party reproduces them. The community’s counterweight to the claim isn’t the benchmark scores, it’s the clock: as we’ll see below, the same model that “beats Opus” on a coding benchmark is the one people are calling too slow to actually code on locally.

For a neutral third-party data point: llm-stats.com has Qwen3.8-27B beating Llama 3.3 70B on GPQA while losing on raw text benchmarks — a plausible profile for a VLM-optimized dense model versus a text-first one.

Real-World Speeds by Hardware Class

Benchmarks measure capability; tok/s is what you feel. These are community-reported figures, not official numbers — treat them as directional:

Hardware Setup Throughput Source
RTX 6000 Pro 96GB (475W limit) BF16, vanilla vLLM 0.27.1 28.64 ± 0.02 t/s (tg32) HumanDrone8721, r/LocalLLaMA
RTX 6000 Pro 96GB (475W limit) official FP8, vanilla vLLM, MTP off 50.08 ± 0.03 t/s (tg32 @ 0 ctx); 33.10 @ 248K ctx same thread
RTX 3090 24GB Unsloth UD-Q4_K_XL, llama.cpp b10088, speculation off 41.49 tok/s (tg128); +254 MiB peak VRAM vs 3.6-27B InsiderLLM
DGX Spark (Blackwell) vLLM, NVFP4 ~33 t/s (1.2–1.5x over FP8) danielhanchen, HN
Apple M4 Pro 48GB mlx-dspark v0.10.0 (speculative, lossless-verified) 8-bit: 8.3 → 20.3 tok/s (2.45x mean); 4-bit: 25.3 tok/s (1.74x) A-Rahim, r/LocalLLaMA

Three things jump out:

  1. Every verified number above ran with MTP off — and that’s the gap in the story. The 6000 Pro thread is vanilla vLLM throughout, and the 3090 bench doesn’t run speculative decoding at all. What is known: the 3.8 GGUF ships the MTP head inside the base file (65 blocks vs 64 in 3.6), it idles at +254 MiB peak VRAM, and the 6000 Pro thread’s OP reports from his own SGLang runs that MTP is “really useful at low context (<128K)” but harmful above that, with MTP3 his best setting — while “SGLang with MTP and NVFP4 has astrocomical high generation values, not sure about precision.” Headline MTP-on numbers for 3.8-27B don’t exist yet; that’s the next data point, not a current selling point.
  2. Blackwell + NVFP4 is the fast path; official FP8 is the everyone-else path. Unsloth’s NVFP4 set (90,924 downloads) targets exactly this: danielhanchen’s DGX Spark report of ~33 t/s at NVFP4, 1.2–1.5x over FP8. The official FP8 variant is the answer for the cards that can’t run NVFP4 — and in the 6000 Pro thread it pulled 50.08 t/s where BF16 managed 28.64, with the OP rating its quality “very high, close to max.”
  3. Q4 on last-gen 24GB cards is the sweet spot — and faster than the complaint thread implies. The 3090 bench shows 41.49 tok/s at UD-Q4_K_XL, within a percent of 3.6-27B at the same quant, at a +254 MiB peak-VRAM cost. The 12–30 t/s numbers people cite in the HN thread are about 8-bit fits and longer contexts — which is what pushes the 16GB crowd toward Q4, where the unsloth GGUF set (867,963 downloads) lives.

Community Reception

Sentiment is overwhelmingly positive. The HN thread (1,213 points, 723 comments) and r/LocalLLaMA are full of superlatives — “game-changer for local AI,” and the phrase “the most important local AI release of 2026” circulating from @rosgluk’s Medium analysis — driven by the combination of Apache 2.0, the agentic benchmark jump, and a GGUF ecosystem that was ready before the weights were.

“These are massive improvements — and something you can actually run on a laptop.” — chvid, HN

“I hope there also will be a new ~10b variant.” — UncloOxidant, HN

The criticisms, all of them, are about speed rather than intelligence:

  1. 8-bit throughput on consumer GPUs. The sharpest version comes from SwellJoe in the HN thread: at 12–30 t/s, “if all you do with AI is coding, you’re better off doing it in the cloud.”
  2. Quant degradation. A recurring Reddit complaint that the new generation loses speed to 3.6-27B at the same quant — though the InsiderLLM 3090 bench measures the gap at under a percent, so the spread of community reports is wide and the kernel-friendliness story is unproven.
  3. Methodology, not the model. A side-debate in the HN thread questions whether KL-divergence is a valid proxy for agentic capability in quant evaluations — i.e., are we measuring the right thing when we score quants?

The most honest summary from the thread is also SwellJoe’s, in the same breath as the criticism:

“Qwen3.8-27B is the tipping point… probably slow enough I won’t use it, but it’s looking really promising.”

“I’d rather pay $100/month for a faster model, even if Qwen 3.8 turns out to be smart enough.”

That tension — capability that beats frontier-class models on paper, latency that doesn’t — is the actual state of 27B-class local AI in August 2026.

What It Means vs the 2.4T Flagship

The most striking number in this whole story isn’t a benchmark. It’s a ratio:

Metric Qwen3.8-2.4T-A95B Qwen3.8-27B
Likes 950 9,497 (~10x)
Downloads 6,381 91,917 (~14x)
Architecture MoE, 512 experts (10/tok) Dense
Modality Text only Vision-language
License other apache-2.0

A 2.4T-parameter MoE flagship with 512 experts is the model a lab builds to win leaderboards. The 27B is the model a person runs on a laptop. The community’s download vote says the second kind of model is what people actually want — capability at a size that fits in a machine you can hold. The 2.4T’s other license and datacenter footprint make the contrast less “bigger is better” and more “bigger is for the lab.”

There’s a second telling number: the official FP8 variant has pulled 123,157 downloads against 91,917 for the base model — the quantized official weights are out-downloading the full-precision ones, with the card claiming “nearly identical” performance. Quant-first isn’t a community workaround anymore; it’s the official distribution channel. The base model is now the reference point, not the product.

Predictions

What the community is betting on, in rough order of plausibility:

  • Qwen3.8-Max open weights “next week” — @rosgluk’s Medium piece puts the Max’s open weights at “next week.” The 3.8 family so far is the 27B dense + the 2.4T-A95B MoE; a Max with open weights would complete the pattern. (Treat as a rumor until the card exists.)
  • A ~122B MoE mid-size — the most-upvoted wish in the HN thread (“I’m hoping for a 3.8-122B MoE” — UncleOxidant). It would slot the family between 27B and 2.4T for people with multi-GPU rigs.
  • A ~10B variant — requested by UncloOxidant in the same thread. On the small end: there’s no official 4B from the Qwen org yet (an HF API search for Qwen3.8-4B returns no Qwen-org repo) — the 4B variants in circulation are community-distilled, like Ma7ee7/Qwen3.8_4B_Distilled_GGUF, which has already pulled 138,874 downloads.
  • Apple-specific optimization — A-Rahim’s mlx-dspark post (up to 2.45x mean on an M4 Pro 48GB, verified lossless) shows Apple moving fast but from behind; the top comment’s pushback — that the multiplier may collapse once context hits 20K without prefix caching for hybrid GDN models — is the real question.
  • The quant race continues — with 868K GGUF downloads, the community’s next job is obvious: close the kernel gap so 3.8 matches 3.6-27B’s speed at the same quant.

Watch List

  • FP8 vs GGUF velocity — if the official FP8 set keeps out-downloading the base, expect the “quantized official” to become the standard release shape for the whole family.
  • Qwen3.8-Max card — the @rosgluk Medium rumor has a “next week” timestamp; the Qwen org page will tell first.
  • MTP-on numbers for 3.8-27B — every verified benchmark above ran with speculation off; the 6000 Pro thread’s OP has SGLang+MTP+NVFP4 follow-ups queued, and those numbers will decide whether MTP becomes a headline for this model.
  • Third-party verification of the Opus 4.6 Max claim — until an independent harness reproduces Qwen’s agentic head-to-head, it stays a claim.

Sources:

Scanned: August 15, 2026 Author: AI Updates