Guide: N-gram Embedding — 51B parameters that multiply nothing

Qwen3.8-Flash-Next hides 51 billion parameters in a 20-million-row lookup table. Reading it costs a memory access, not a matrix multiply — a third way to scale a model that changes what your hardware bill looks like. What it is, why it offloads where experts can't, and what it means for a local multi-GPU build.

architecture primer Qwen3.8-Flash-Next / "Qwen4Exp" 180B total, ~6B active

The short version

N-gram embedding in one paragraph An n-gram embedding layer is a lookup table inside the model weights: it maps short sequences of adjacent tokens (pairs and triples) to learned vectors. Qwen3.8-Flash-Next holds 20,000,000 such entries at layer 2 of its 48-layer stack — 51 billion parameters next to a 125B backbone. When a token arrives, the model indexes the table by the short phrases ending at that position and folds a handful of vectors (a few kB) into the residual stream. Nothing is multiplied, so the 51B adds capacity without arithmetic — and because the access is a single row read per token, the table can live in system RAM or even an SSD without killing speed. The result is a model that "reads like a 180B and serves like a 6B."

First: two different things called "n-gram"

You may have heard "n-gram" mentioned in two unrelated contexts. Don't mix them up:

N-gram embedding (this page)N-gram draft (speculative decoding)
What it isA learned lookup table that is part of the model's weightsAn inference-time trick where the engine proposes tokens copied from context/prompt history
Where it livesIn the model file (the 51B table)In the engine (llama.cpp / vLLM code), no model support needed
CostMemory (VRAM/RAM/SSD) — almost no computeA little compute; works on any model
QualityTrained into the model — it's how the model thinksLossless (verify-accept), pure speed
ExamplesQwen3.8-Flash-Next; DeepSeek's "Engram" moduleContext-matching drafting in speculative decoding

The MTP guide covers the second family. This page is about the first — and why it matters for hardware planning.

The problem: every scaling axis before it cost compute

There were two familiar ways to make a language model bigger:

Qwen3.8-Flash-Next's n-gram table is a third axis. Language contains a lot of information that is local and fixed: which word follows which in common phrases, which token pairs never co-occur. Storing those facts in weights that participate in every matrix multiplication is expensive. A table can hold them at the cost of the memory they occupy — and a table is addressed by a hash of the last few tokens, so the address exists the moment the tokens exist. The network never multiplies the table; it gathers a handful of rows and folds them into the stream.

What the table holds

In Qwen3.8-Flash-Next the table sits at layer 2 — near the bottom of a 48-layer stack — and indexes 20,000,000 bigrams and trigrams (pairs and triples of adjacent tokens), each mapped to a learned vector. That's 51 billion parameters, roughly the size of a big dense model, doing the work of a dictionary lookup. The operation is an array index: a dense layer of the same size would add 51 billion multiply-accumulate operations per token; the table adds a handful of memory reads.

The community summary of the division of labor is one line: experts do reasoning, n-grams do recalling. The table absorbs the common local patterns, so the early layers stop wasting depth rebuilding them and the deeper stack is free to reason. Of course recall without reasoning is a memory-fetching machine, not a model — the table is an addition to the MoE backbone, not a replacement.

Qwen3.8-Flash-Next: the first model to ship it

Released 24 Aug 2026 under the qwen4_exp tag, Flash-Next is an experimental preview of the architecture that will underpin Qwen4. The full parameter bill:

ComponentParametersRole
MoE backbone (48 layers, 512 experts, 10 routed + 1 shared active)125BReasoning — ~6B activated per token
N-gram embedding table (20M bigram/trigram rows, layer 2)51BRecall — lookup only, no arithmetic
MTP module (1 layer, multi-step trained)4BSpeculative decoding — see the MTP guide
Total180B~6B active per token

It pairs the table with other Qwen4-architecture changes: hybrid attention (Gated DeltaNet + Qwen Sparse Attention, which operates at micro-block level to cut long-context latency), gated residual streams, and a 262K native context extensible to 1M. The official FP8 checkpoint is 172.78 GiB. Quality-wise it punches far above its active count — e.g. 62.5 on SWE-bench Pro (vs 61.7 for the 27B-dense Qwen3.8-27B, which activates 27B), 91.7 GPQA Diamond and 91.9 LiveCodeBench v6. The managed "Qwen3.8-Flash" API model is the production version of the same architecture.

The three scaling axes, compared

AxisCompute per tokenMemoryCan it offload?
Dense layersEvery parameter multipliesFull, residentNo
Mixture of expertsOnly routed experts multiplyFull, residentPoorly
N-gram embeddingA lookup — no multiplicationFull, but movableYes, to host RAM / SSD

Flash-Next uses all three at once: 125B across experts, 51B in the table, 4B in MTP. The offload column is the one that changes your hardware bill. Offloading works because the usage sparsity is immense — one access per token, one embedding vector per access — and prefetching hides the latency. You never get a cache hit and it doesn't matter; the working set per step is a few kilobytes.

What this means for your hardware

The memory bill splits into two

The 172.78 GiB FP8 checkpoint is the headline number, but it's not all VRAM. The vLLM recipe keeps the 51B table in host RAM (at least 51 GB of system RAM plus headroom, with background prefetching; NVIDIA-only in the initial release) and treats it as optional under tensor/expert parallelism. The community's framing: a ~176B model becomes 125B in fast memory + 51B on slow media — with SSD storage in the mix once the table's long tail of rare patterns is stratified out. DeepSeek's equivalent "Engram" module makes the same point explicitly with a multi-level hierarchy: hot rows in HBM, the bulk in DRAM, the rare tail on NVMe.

The compute bill stays small

Only ~6B parameters are active per token, so token generation lands in small-MoE territory — the table contributes capacity without adding arithmetic. (Early adopters do report that prompt-processing speed is not the strong suit yet; the architecture is a preview, and engines are still catching up.)

The multi-GPU shape is constrained

One hard constraint: pipeline (layer) parallelism does not work — the n-gram layer has no pipeline-parallel implementation in the initial release. The supported route is single-node tensor or expert parallelism. That aligns neatly with the multi-GPU guide's conclusion that tensor splitting beats layer splitting: a 3–4× 24 GB NVLink box (72–96 GB) plus a large-RAM motherboard is the natural local shape for this model class, not a 2×3090 PCIe pair running a layer split. Expect the 51B table to live mostly in system RAM/SSD, so a big, cheap RAM/SSD budget matters more than it used to.

The 20–25% rule

There's a limit to how much capacity you can move into the table before it stops helping. The community read of the current frontier: giving the n-gram table roughly 20–25% of total parameters is where the trade-off peaks — enough to absorb local patterns without turning the model into a lookup machine. Flash-Next's 51/180 ≈ 28% is right in that neighborhood.

Status for local runners

The official release targets transformers / vLLM / SGLang. GGUF quants already exist (e.g. the unsloth GGUF repo), so llama.cpp support is on the way — but this is a brand-new architecture: before planning a build around it, check current engine support and benchmark with your own setup. The table's offload path is where the surprises will be.

Will this spread?

Qwen explicitly frames Flash-Next as a preview of the Qwen4 architecture, so the table is likely a permanent fixture of the next family rather than a one-off experiment. DeepSeek has shipped the same idea under a different name — Engram — with the module shared across multiple layers instead of Qwen's layer-2-only placement, so at least two frontier labs are converging on "a big lookup table as a scaling axis." What we don't have yet is an ablation separating the table's contribution from the model's other changes; the technique arrived with the model, not with an isolated study.

For anyone building hardware: the practical effect is narrow and real. A class of model that reads like 180B and serves like a 6B-A model — provided you can hold its footprint, and increasing parts of that footprint are starting to live outside the GPU.

Sources

← Multi-Token Prediction (MTP) next: multi-GPU builds →