How fast is Qwen3.8-27B really? What local benchmarks actually show

"Qwen3.8-27B gets 40 tokens a second" and "Qwen3.8-27B gets 127 tokens a second" are both true statements, about the same model, sometimes on the same GPU. The gap isn't the hardware. It's almost entirely how much engineering work went into the serving setup, and most people never see the high end of that range.

One model, four very different numbers

Independent local benchmarks for Qwen3.8-27B, run on a single 24GB-class consumer card, land anywhere from the mid-30s to the low 40s tokens/second with a default setup, up past 120 with a heavily customized one. NeverMetered's own measured speed for the same model on its hosted servers is 42 tokens/second on a single stream, published on the model page alongside the benchmark scores. That's not a coincidence worth hiding: it sits right in the default-setup range, not the tuned one, and it's worth explaining exactly why.

What a default local setup actually gets you

Run Qwen3.8-27B with a common, off-the-shelf local inference stack on a 24GB-class card and independent tests report just over 40 tokens/second at short context, dropping to somewhere in the mid-30s once the context grows to 64K tokens. That's the number most people running this model locally actually see: whatever a popular inference tool does by default, with no custom patches, no speculative decoding, no requantization work.

What tuning buys you, and what it costs

The higher numbers are real, and they come from real engineering, not a different GPU. One documented technique, speculative decoding via multi-token prediction (MTP), lifts the same default setup from about 41.6 to 66.4 tokens/second on the same card, a 59.8% jump. That's a meaningful, achievable upgrade for someone willing to enable and tune it.

The high end of the range, 114 to 127 tokens/second single-stream, comes from a much deeper project: a patch series against a specific inference engine build, a requantized model head, a calibrated draft vocabulary for speculative decoding, and split-KV verification attention, published as an open GitHub project specifically to reach that number on one 24GB-class card. It's real and reproducible if you follow the recipe. It is also not what you get by installing the tool and running it, and a mainstream hardware outlet covering this same model reached a similar conclusion independently: raw VRAM capacity isn't the bottleneck, the inference engine and how well it's tuned is.

Where hosted sits, and the actual tradeoff

A shared, multi-tenant hosted service is architecturally closer to the default case than the tuned one, and that's a fair thing to be upfront about. NeverMetered's 42 tokens/second single-stream figure isn't the theoretical ceiling for this model, the custom-tuned local setup runs meaningfully faster on the same class of card. What hosted setup trades for that: no patch series to apply, no model requantization to run yourself, and real parallel capacity instead of one exclusive stream, up to 8 concurrent requests per server rather than a single GPU serving one session at a time. If you're the kind of person who'd actually build and maintain the tuned setup, the honest advice is to do it, you'll likely beat a shared hosted service on raw single-stream speed. If you'd rather not run an inference stack at all, current plans show what the flat-rate version costs instead.