16Adding a model

The engine serves big models such as Qwen3.8-27B, a hybrid of Gated DeltaNet and attention layers, and Qwen3.8-Flash-Next, a mixture of experts on the same hybrid (Chapter 17). The small models beside them (the Qwen3.5 narrator and shadow reader) run on the 27B’s forward code. Flash-Next was the first model added through the adapter path below. This chapter lists the seams where a new model plugs in, what each one expects, and the tests a new model must pass before its numbers mean anything. Some of the engine is still specific to this family, and the last section says which parts.

16.1 The adapter path

A new architecture brings a package under engine/models/<name>/, one registry entry (engine/models/__init__.py) and data files, and no edit anywhere else. The methods read a model only through its adapter. tests/test_load_shared.py fails if code under server/ names a model or asks an engine what it is. Five layers depend only downward (Figure 16.1). The upper three exist, the formats as today’s Linear protocol, and so does the server’s one loader. Four pieces are still design: format plugins (ENG-253), a kernel registry (ENG-251), device profiles (ENG-255) and the mesh (ENG-259).

An adapter is the engine object with two sets of verbs. The hot verbs run every round and are the engine’s own bound methods, never a wrapper, so graphs capture the same launches: forward, forward_tree, forward_block, commit_tree, rollback_to, picks, hidden_post_norm, kv.length and reset. The cold verbs run around a cache copy or at load: caps (what the server may ask: caches, side slots, the cache grid, the shadow reader, joint verify), cost() (state bytes and the prefill workspace), cache_state(), resume_at(n), settle(), discard_pending(), trim_pending(n), manifest() (what its stored rows depend on, for the disk tier’s fingerprint), memory_parts(), resident_files() and precapture(). The 27B’s cold verbs are installed on its class by engine/models/qwen3_5/adapter.py, so engine/model.py keeps its bytes.

cache_state() returns a ModelState: sections, the recurrent state by name and copied whole, and views, the position-indexed buffers with their token axis and sliced by length. Every tier keeps a sequence in that form, and a signature over names, shapes and dtypes refuses a restore across models.

A new model then needs, in order: a config reader that refuses unknown fields; a reader for its checkpoint and a Linear object for each weight; a reference forward checked layer by layer against the published implementation; the engine with the hot verbs, every verified row equal to the decode step’s row; the cold verbs; a draft head if the checkpoint ships one (the shared MTP drafter asks for depth, reset, prime, capture_all, launch, read, views and resume); a server builder serve(a, ctx); the registry entry; and its data files: a StairCut price file per profile and device, the weight profiles (ops/profiles/<name>.env) and the Setup Guide’s registry entry (models/<id>/model.yaml, described in docs/setup.md).

16.2 The seams

Config.

engine/config.py::load_config reads config.json into a TextConfig with every size the engine needs, from the layer count to the rotary settings, and answers which layers are recurrent. It accepts a vision-language config (the text model under text_config) and a text-only one. A new family either fits TextConfig or extends it; the rest of the engine asks TextConfig, never the raw JSON.

Layout.

engine/loader.py::Layout says how a checkpoint names its tensors, in one place. The engine addresses every tensor by a canonical name, the language model’s own naming with the checkpoint’s wrapper prefix taken off. The layout lists the prefixes to strip, the tensors the language model never reads (such as a vision tower), where the checkpoint keeps its prediction head, and the projections that may be stored as FP8 pairs or as plain matrices. tests/test_layout.py pins the served checkpoint’s key set: 1,250 language-model keys read, 333 vision keys skipped.

Weight formats.

Every weight the engine multiplies by implements one small interface (engine/linear.py): a shape, the bytes read per use, and matmul. FP8Block, NVFP4Block, FP8Head and BF16Block implement it. The model calls w.matmul(x) and never asks which format it holds. A new format is a new class with those three members, plus a kernel that keeps the row-invariance rule (Chapter 10).

Settings.

engine/settings.py is the registry of every QWEN38_* switch, read only through SETTINGS.get(name). EngineSettings(env, prefix) gives the same reads over another mapping or another prefix.

What a drafter reads.

A drafter reads parts of the target: hidden states at some layers, the embedding, the head. Each drafter declares these through Drafter.requires(): the hidden size, the tap layers, the target tensors it reads by name, and the vocabulary size. At load the engine compares them with the target and refuses a mismatch by name, so a DFlash2 drafter trained for another model fails there instead of drafting garbage. A new model needs its own block drafter; the lookup drafter works for any model once its corpus is rebuilt with the new tokenizer (tools/build_corpus.py). The lookup stores record the tokenizer’s sha256 and refuse to open under another tokenizer.

The tokenizer and the chat template.

The server renders prompts with the model’s own chat template and reads the model’s own tool-call and reasoning formats. A model with another format needs its own parser behind the same OpenAI output.

16.3 The tests a new model must pass

In this order, each on the new model:

1. tools/refcheck.py: argmax agreement with the published reference implementation on teacher-forced positions, the reference in its own process because only one model fits on the board at a time. On Qwen3.8 the engine agrees on 99.25 % of 400 positions and on all 242 confident ones (measured).
2. tools/quality_gate.py for every quantised weight set (Chapter 9).
3. tools/verify_spec.py, the agreement gate with its near-tie rule, with every drafter (Chapter 12).
4. The CPU suite, including tests/test_refactor_gate.py. That test records the exact bytes of a tiny model’s run (every logit and every byte of state after a prefill, a decode, a block verify, a rollback, a tree verify and a commit) and fails if any byte moves. A change that is supposed to change arithmetic re-records the fixture and says why.
5. ops/gate.sh and the client smoke set before any number is published (Chapter 15).

Flash-Next took its own form of these gates: the one loader (tests/test_load_shared.py), every cache tier against a cold prefill (tests/test_cache_any_model.py and its server check), a fingerprint that moves with its kernels and tables and with nothing else (tests/test_fp_v3.py), the load-time self-check of every verify width (3,936 rows bit for bit on the 3.05 pack), and speculative against plain decoding in the engine and through the server (tools/fn_spec_check.py, tools/fn_server_check.py --compare).

16.4 What is still specific to Qwen3.8

• engine/model.py implements the 27B’s forward and engine/models/flashnext/ Flash-Next’s. A new family needs its own layer code behind the same verify, commit and cache interfaces; Flash-Next routes its experts inside the verify, and every row reads its own experts.
• The recurrent-layer kernels assume Gated DeltaNet’s state shape, heads of 128 by 128.
• Tile tables and router prices are measured for each model’s shapes on the GB10. A new model or a new board needs its own sweep and its own price file; the kernel registry and device profiles that would make a board data are planned, not built.
• Some modules read their switches into globals at import, so two engines with different settings cannot share one process yet.
• Runtime names still carry the old project name, from the QWEN38_ and QSE_ prefixes to the Qwen38Engine class.

The Qwen3.5 small models already run on the engine’s own forward: the narrator and the shadow reader load them through engine/generative/qwen35.py, the same layer code, because they share Qwen3.8’s architecture family and its tokenizer (Chapter 24).