Qwen3.8-Flash-Next hides 51 billion parameters in a 20-million-row lookup table. Reading it costs a memory access, not a matrix multiply — a third way to scale a model that changes what your hardware bill looks like. What it is, why it offloads where experts can't, and what it means for a local multi-GPU build.
You may have heard "n-gram" mentioned in two unrelated contexts. Don't mix them up:
| N-gram embedding (this page) | N-gram draft (speculative decoding) | |
|---|---|---|
| What it is | A learned lookup table that is part of the model's weights | An inference-time trick where the engine proposes tokens copied from context/prompt history |
| Where it lives | In the model file (the 51B table) | In the engine (llama.cpp / vLLM code), no model support needed |
| Cost | Memory (VRAM/RAM/SSD) — almost no compute | A little compute; works on any model |
| Quality | Trained into the model — it's how the model thinks | Lossless (verify-accept), pure speed |
| Examples | Qwen3.8-Flash-Next; DeepSeek's "Engram" module | Context-matching drafting in speculative decoding |
The MTP guide covers the second family. This page is about the first — and why it matters for hardware planning.
There were two familiar ways to make a language model bigger:
Qwen3.8-Flash-Next's n-gram table is a third axis. Language contains a lot of information that is local and fixed: which word follows which in common phrases, which token pairs never co-occur. Storing those facts in weights that participate in every matrix multiplication is expensive. A table can hold them at the cost of the memory they occupy — and a table is addressed by a hash of the last few tokens, so the address exists the moment the tokens exist. The network never multiplies the table; it gathers a handful of rows and folds them into the stream.
In Qwen3.8-Flash-Next the table sits at layer 2 — near the bottom of a 48-layer stack — and indexes 20,000,000 bigrams and trigrams (pairs and triples of adjacent tokens), each mapped to a learned vector. That's 51 billion parameters, roughly the size of a big dense model, doing the work of a dictionary lookup. The operation is an array index: a dense layer of the same size would add 51 billion multiply-accumulate operations per token; the table adds a handful of memory reads.
The community summary of the division of labor is one line: experts do reasoning, n-grams do recalling. The table absorbs the common local patterns, so the early layers stop wasting depth rebuilding them and the deeper stack is free to reason. Of course recall without reasoning is a memory-fetching machine, not a model — the table is an addition to the MoE backbone, not a replacement.
Released 24 Aug 2026 under the qwen4_exp tag, Flash-Next is an experimental
preview of the architecture that will underpin Qwen4. The full parameter bill:
| Component | Parameters | Role |
|---|---|---|
| MoE backbone (48 layers, 512 experts, 10 routed + 1 shared active) | 125B | Reasoning — ~6B activated per token |
| N-gram embedding table (20M bigram/trigram rows, layer 2) | 51B | Recall — lookup only, no arithmetic |
| MTP module (1 layer, multi-step trained) | 4B | Speculative decoding — see the MTP guide |
| Total | 180B | ~6B active per token |
It pairs the table with other Qwen4-architecture changes: hybrid attention (Gated DeltaNet + Qwen Sparse Attention, which operates at micro-block level to cut long-context latency), gated residual streams, and a 262K native context extensible to 1M. The official FP8 checkpoint is 172.78 GiB. Quality-wise it punches far above its active count — e.g. 62.5 on SWE-bench Pro (vs 61.7 for the 27B-dense Qwen3.8-27B, which activates 27B), 91.7 GPQA Diamond and 91.9 LiveCodeBench v6. The managed "Qwen3.8-Flash" API model is the production version of the same architecture.
| Axis | Compute per token | Memory | Can it offload? |
|---|---|---|---|
| Dense layers | Every parameter multiplies | Full, resident | No |
| Mixture of experts | Only routed experts multiply | Full, resident | Poorly |
| N-gram embedding | A lookup — no multiplication | Full, but movable | Yes, to host RAM / SSD |
Flash-Next uses all three at once: 125B across experts, 51B in the table, 4B in MTP. The offload column is the one that changes your hardware bill. Offloading works because the usage sparsity is immense — one access per token, one embedding vector per access — and prefetching hides the latency. You never get a cache hit and it doesn't matter; the working set per step is a few kilobytes.
The 172.78 GiB FP8 checkpoint is the headline number, but it's not all VRAM. The vLLM recipe keeps the 51B table in host RAM (at least 51 GB of system RAM plus headroom, with background prefetching; NVIDIA-only in the initial release) and treats it as optional under tensor/expert parallelism. The community's framing: a ~176B model becomes 125B in fast memory + 51B on slow media — with SSD storage in the mix once the table's long tail of rare patterns is stratified out. DeepSeek's equivalent "Engram" module makes the same point explicitly with a multi-level hierarchy: hot rows in HBM, the bulk in DRAM, the rare tail on NVMe.
Only ~6B parameters are active per token, so token generation lands in small-MoE territory — the table contributes capacity without adding arithmetic. (Early adopters do report that prompt-processing speed is not the strong suit yet; the architecture is a preview, and engines are still catching up.)
One hard constraint: pipeline (layer) parallelism does not work — the n-gram layer has no pipeline-parallel implementation in the initial release. The supported route is single-node tensor or expert parallelism. That aligns neatly with the multi-GPU guide's conclusion that tensor splitting beats layer splitting: a 3–4× 24 GB NVLink box (72–96 GB) plus a large-RAM motherboard is the natural local shape for this model class, not a 2×3090 PCIe pair running a layer split. Expect the 51B table to live mostly in system RAM/SSD, so a big, cheap RAM/SSD budget matters more than it used to.
There's a limit to how much capacity you can move into the table before it stops helping. The community read of the current frontier: giving the n-gram table roughly 20–25% of total parameters is where the trade-off peaks — enough to absorb local patterns without turning the model into a lookup machine. Flash-Next's 51/180 ≈ 28% is right in that neighborhood.
The official release targets transformers / vLLM / SGLang. GGUF quants already exist (e.g. the unsloth GGUF repo), so llama.cpp support is on the way — but this is a brand-new architecture: before planning a build around it, check current engine support and benchmark with your own setup. The table's offload path is where the surprises will be.
Qwen explicitly frames Flash-Next as a preview of the Qwen4 architecture, so the table is likely a permanent fixture of the next family rather than a one-off experiment. DeepSeek has shipped the same idea under a different name — Engram — with the module shared across multiple layers instead of Qwen's layer-2-only placement, so at least two frontier labs are converging on "a big lookup table as a scaling axis." What we don't have yet is an ablation separating the table's contribution from the model's other changes; the technique arrived with the model, not with an isolated study.
For anyone building hardware: the practical effect is narrow and real. A class of model that reads like 180B and serves like a 6B-A model — provided you can hold its footprint, and increasing parts of that footprint are starting to live outside the GPU.