36Changelog summary

CHANGELOG.md in the repository lists every release with its measurements. This chapter summarises it. Every release is held to the one before it on the raw path by the identity gate (the same logits and state, bit for bit, on a fixed scenario), except where an entry says otherwise, and each one’s speed is checked on the benchmark row against the release before it, in the same hour. The gate fixes the arithmetic, not the tree shapes the length router picks from timings, so the served text can part at a near tie between two runs of either release (Section 12.3).

release	date	what it brought
0.1.0-rc1 to rc4	2026-09-18 to 09-25	the engine, the length router, release of the losing drafter, serving defaults for long work; rc4 is the first release measured in two store views
0.1.0-rc5 to rc7	09-26 to 09-27	sampled requests on the draft tree, the in-process block A/B, tool calls read in three forms and streamed as argument deltas
0.1.0-rc8 to rc10	09-27 to 09-28	the resident prefix, live activity on the dashboard, the FP8 and balanced profiles, the first seams for other models
v0.2.0-staircut	09-29	StairCut, on in the served profile: the paper’s release
v0.2.1	09-29	the dashboard login off by default; no decode path changed
v0.3.0	09-30	image input; the text path unchanged
0.4.0	10-06	the conductor, memory notes, the caches for long agent sessions, parallel requests (off by default), and the dashboard as a standalone proxy in front of other engines and cloud APIs; not published, 0.4.7 replaces it
v0.4.7	10-08	0.4.0 plus the package library, Regenerate, long Open WebUI chats back from the NVMe, the compressed session spill and a daily NVMe write budget
v0.4.8	10-08	two Open WebUI chats beside each other (a side slot keeps its conversation), a chat whose retrieval block moved resumes at its end, the park keeps the end anchor, the router’s notes stay live by their message
0.5.0	10-09	weight profiles: NVFP4 or EXL3 at 2.5 to 6.0 bits a weight, one at a time, switched with ops/use-profile.sh
0.5.1	10-10	two dashboard numbers corrected; four parallel requests on the reference box
0.6.0	10-10	one shared KV pool that maps pages as a conversation grows; a request that does not fit waits instead of failing
0.6.1	10-10	the small models give memory back when the pool runs short; the memory split counts committed pages
0.6.2	10-10	decoding no longer stops while another slot prefills; the Perf tab’s live feed fixed
0.7.0	10-11	the setup guide at #/setup with Wren, its assistant; reset levels; setup mode
0.7.1	10-11	Wren drawn as the DGX Spark box; dashboard only
0.7.2	10-11	Wren answers a question in words again; setup guide only
0.8.0	10-13	a second model, Qwen3.8-Flash-Next, from turboderp’s EXL3 packs; the Chat tab with chatd; the cache peek and prewarm
0.8.1	10-13	a new box downloads every file a weight profile needs, with no token; the EXL3 weights public; the engine unchanged
0.8.2	10-13	community weight packs; permission modes in the Chat tab; standalone EXL3 packs of the 27B family; a chat no longer waits minutes for slot 0 while a side slot is free; the model says it is TandemLLM when asked
0.8.3	10-13	repository hygiene for the public release; no code change
0.9.0	10-14	a third model, LFM2.5-24B-A2B: six EXL3 sizes and BF16, the default 3.5 by a stated rule, the lookup drafting without a draft head; Wren beside it
0.9.1	10-14	web search and media playback in the Chat tab; the loop guard, the loop escape and memory notes on in the shipped conductor profiles; Setup Guide fixes; the 27B’s EXL3 profiles fetch only the base tensors they read
0.9.2	10-15	install fixes: the checksum step reads a download’s globs; after install.sh --setup the 27B’s EXL3 profiles take base-plain
0.10.0	10-15	a fourth model, Nemotron 3.5 Bolt 74B-A9B: seven EXL3 sizes, NVIDIA’s IQ2 GGUF and its LiLiCorr drafter; a 1 GiB limit for each Chat tab sandbox; attachments never inlined past the window
0.11.0	10-16	a fifth model, Nemotron 3.5 Super VL 120B-A12B: EXL3 packs at 4.0 and 2.0, NVIDIA’s NVFP4 release read as stored, its LiLiCorr drafter

36.1 0.11.0, Nemotron 3.5 Super VL

0.11.0 adds a fifth model, NVIDIA’s Nemotron 3.5 Super VL 120B-A12B (Section 19.5). The other models’ text and speed do not change, and their saved conversations carry over.

Super VL.

Our EXL3 packs at 4.0 and 2.0 bits a weight and NVIDIA’s NVFP4 release, read as stored (NVFP4 experts, FP8 Mamba2 and attention projections and head), each with NVIDIA’s LiLiCorr drafter for Super VL beside the lookup. The 4.0 pack is the default: KL 0.049 against BF16 (NVIDIA’s NVFP4 0.041), 31.1 tok/s for one request (NVFP4 19.6), and with LiLiCorr and the lookup 53.6 tok/s on code and 101.0 on edit asks (the mean of 3 runs). Text only: the vision tower is not served yet.

36.2 0.10.0, Nemotron 3.5 Bolt

0.10.0 adds a fourth model, NVIDIA’s Nemotron 3.5 Bolt 74B-A9B (Chapter 19). The other models’ text and speed do not change, and their saved conversations carry over.

Bolt.

The NemotronH engine (Mamba2, attention without positional encoding, relu$^2$ experts behind a latent projection), our EXL3 packs at 2.0 to 6.0 bits a weight, NVIDIA’s IQ2 GGUF read as stored, and NVIDIA’s LiLiCorr drafter beside the lookup. The 2.0 pack is the default: KL 0.170 against BF16 (the GGUF 0.259), 43.3 tok/s for one request, and with LiLiCorr and the lookup 65.7 tok/s on prose and 102.8 on code (the mean of 3 runs). A verify block’s EXL3 projections decode each trellis tile once for 16 rows, every output byte the one-row call’s. Text only: the image tower is not served yet.

Chat tab.

Each chat’s sandbox has 1 GiB of memory, and a call the kernel kills at the limit says so. An attachment is never inlined past the window: every attachment is saved under the chat’s /workspace/uploads/, and a text file past a quarter of the free window goes as one line with its path. A tool call no longer waits without end for its sandbox. POST /v1/cache/release gives back memory the engine holds idle.

Saved conversations.

Fingerprint v2 leaves out modules under engine/ that only another model imports, so the 27B’s code fingerprints stay those of 0.9.x; the first Flash-Next or LFM2.5 start adopts the files 0.9.x wrote, once.

36.3 0.9.2, install fixes

0.9.2 fixes two install bugs on a new DGX Spark. The engine, the conductor and the dashboard do not change.

The checksum step.

fetch_weights.py compared a download’s file globs to file names literally. A model whose list is all globs, Qwen3.5-0.8B for the shadow reader or Qwen3.5-2B for the narrator, checked no file and failed with “0 files checked: FAILED”, and Laya’s files under multilingual/ and typed-decisions/ went unchecked. The globs now select the files.

The EXL3 base after a setup install.

The installer wrote QWEN38_MODEL, the Hugging Face cache’s base folder, after the weight profile, so the 27B’s EXL3 profiles kept that empty folder instead of base-plain and use-profile.sh refused to switch. The installer’s own keys now come before the profile, an EXL3 profile takes base-plain when the folder holds no snapshot, and a standalone community pack reads its own folder. An install with HF_HOME writes the cache to serve.env as HF_HUB_CACHE. To upgrade, run the 0.9.2 installer again: it rewrites serve.env and the scripts beside it.

36.4 0.9.1, web search, media and the loop guard

0.9.1 changes the Chat tab, the conductor’s shipped profiles and the Setup Guide. The decode path does not change: under engine/ only the small models’ slot changed, so that a failed load is tried again.

Chat tab.

A built-in web_search tool with DuckDuckGo and SearXNG providers, set in the chat settings. Video, audio and PDF files open in a player or the browser’s PDF viewer, in the Files pane and the present card; any other binary file shows its size and a Download button. chatd streams a workspace file with byte ranges (206, 416, ETag) and no 2 MiB cap.

The loop guard.

The same-call guard (C3) is on in the shipped coding, chat-app, hermes, overnight and chat profiles. It counts only the current user turn, treats a tool result that took the client 10 s or more as a wait, and holds calls only on a request whose turn already ends with the repeat. After a repeated or near-repeated call the session samples with the model card’s settings (the checkpoint’s generation_config.json, a fresh seed) until it makes progress, then decodes greedy again; a client’s own sampling wins. Output changes on those turns only. --reasoning-effort none and off turn thinking off for every request that says nothing about thinking.

Memory notes.

The shipped chat, coding, hermes and chat-app profiles have memory: on. A one-time migration adds the switch to each stored profile without it; an explicit value stays. The memory card tells the model the notes are its memory between chats, a profile with memory off tells it so, and the Context panel shows a turn’s notes.

Fixes.

The Setup Guide’s install page keeps a finished job final and its whole log after a reload. The change list names the keys a model switch drops from run/setup.env. The fetch step downloads the models of the tools the selection turns on, and a failed shadow reader load is tried again after 300 s in an idle window. The 27B’s EXL3 profiles read their base from base-plain, fetched by HTTP range requests: 3.54 GB instead of 30.41, so a fresh exl3-6.0 downloads 36.48 GB, not 63.34. The dashboard’s service worker shows the cached shell when the reverse proxy answers 502, 503 or 504, and reloads once /health answers.

36.5 0.9.0, LFM2.5-24B-A2B

0.9.0 adds a third model, LiquidAI’s LFM2.5-24B-A2B, the first here that is not a Qwen (Chapter 18). The Qwen models’ text and speed do not change: the shared EXL3 kernels learned half-integer rates, and SAME_ROWS names their new bytes as the old ones, so the first Qwen start adopts its saved conversations once.

Profiles.

Liquid’s BF16 checkpoint and six EXL3 sizes of our own, 2.5 to 6.0 bits a weight, each at the model’s whole 131,072-token context with three slots. The decode step runs at 188.2 tok/s on 2.5, 153.5 on 3.5, 106.9 on 6.0 and 44.8 in BF16 (measured, 2026-10-14), with each pack’s own dense splits. The default is lfm25-exl3-3.5: the fastest profile in chat as it serves whose chat top-1 agreement with BF16 is at least 85 % (143.7 tok/s at 86.7 %; 3.0 is faster at 83.6 %). An exllamav3 -hq 4.0 was slower than 4.0 and worse than 4.5, and is not offered.

The lookup without a draft head.

A round with no lookup match is the plain step, priced as one; the lookup’s confidence carries its match length; every profile has its own StairCut price file. With the store off, edit and copy asks on 4.0 ran at 291.1 and 326.4 tok/s against 141.3 and 141.9 plain, every text equal.

The conductor.

On for LFM2.5 by default, with Laya and the narrator, through the model’s fit to the router: Laya alone answers the class question, a turn nobody classified gets nothing added, and a class sets the level and budget with only its core skill in the note. On 2.5, four short chat asks thought 2,234 reasoning tokens against 2,155 raw and 9,098 under the router as written; seven Chat-tab tasks run twice succeeded 14 of 14, as raw did, against 12 of 14 (Section 18.5.1).

Fixes.

A pack stored as one model.safetensors counts as complete in ops/use-profile.sh and the Setup Guide. Reading a slot’s kv.length commits nothing. Attention’s split buffers are kept per attention class. A suffix store of another tokenizer turns the store off instead of stopping the start. The dashboard’s disk tier default reads 16 GB, as the server’s. Wren loads its own Qwen3.5-2B beside LFM2.5 (20 s, 3.62 GiB).

36.6 0.8.3, repository hygiene

Repository hygiene for the public release: no private hostnames, home paths or personal notes in the files; no code change. The engine’s modules are the bytes of 0.8.2, and the text does not change. Comments, docs and tests no longer name a person who gave an instruction, and tests use 192.168.1.x for a LAN address and tandem.example.lan for a host behind a proxy. install.sh installs v0.8.3 by default.

36.7 0.8.2, community weights and permission modes

0.8.2 adds community weight packs, permission modes in the Chat tab and standalone EXL3 packs of the 27B family, and fixes two bugs found on v0.8.0 in Open WebUI on a phone. The engine’s modules are the bytes of 0.8.1.

Community weights.

One JSON file per pack under models/<model>/community/, checked by the registry against models/schema/community.schema.json and gathered in models/community-index.json. The setup guide lists the packs after our profiles as community/<id>, with search and five sort orders. It fetches the index from the public repository when its weights step opens or on Refresh, never on a timer, and falls back to its last copy, then to the one in the release; a 404 shows as the note “no published list yet”. ops/use-profile.sh community/<id> downloads a pack at its pinned commit, checks the hub’s sha256 and runs it with our drafters. An issue form and a check on pull requests come with it. The first entry is ThinkingCap Qwen3.8-27B Kearuga by 0xWhiteMage: EXL3 at 4.0 bits a weight, standalone, 39.6 tok/s prose, 95.9 code and 51.3 chat on 0.8.2 (measured). Chapter 7.

Permission modes.

Each chat in the Chat tab runs in one of three modes, and Auto is the new default: a call that stays inside the chat’s sandbox runs without a prompt, and one that leaves it still asks. Deny rules and plan mode’s read-only list hold in every mode. A tool card that ran without a prompt carries a badge, and every such call is logged as tool.auto_approved. “Always allow” on a chained bash command now saves a rule for every part not yet allowed; before, one chat saved cd 11 times and still got the prompt. Chapter 27.

Standalone EXL3 packs.

A complete BF16-origin mul1 EXL3 pack of the 27B family loads as --model, with no separate base checkpoint (pull request #1 by 0xWhiteMage). The path is text only, and speculative parallel serving still needs our drafters (Section 9.8). The 27B’s own profiles take the path they took before. engine/loader.py’s new bytes count as the old ones in the disk tier’s fingerprint (SAME_ROWS in engine/methods/manifest.py), so the first 27B start adopts v0.8.0’s saved files once, and the 27B’s code fingerprint stays 0.7.2’s.

The slot-0 wait.

A new Open WebUI chat waited 437 s for slot 0 behind another chat’s 496 s generation while both side slots were free. It shared only Open WebUI’s system prompt and tool list with an entry on the NVMe, and the wait for slot 0 had no limit. Now the spill wait needs the request’s own conversation, and every wait for slot 0 ends after 3 s (QWEN38_SIDE_SLOT0_WAIT_S) when a side slot the request fits in is free. /health parallel.max_running is the limit with its reason, not the high-water mark, and the Lanes view says why a request is queued. On the reference box a turn that slot 0’s chat had held moved to slot 2 after its 3.0 s wait, while another chat ran 80 s on slot 0 (measured).

Identity.

Asked who it was, Flash-Next named another assistant and its maker: the reference box served it with the conductor off, so the identity note never went in. The note now says the model is TandemLLM running the served model (Qwen3.8-Flash-Next or Qwen3.8-27B, from the pack’s model card) on the user’s own machine, for every client the conductor serves, and the reference box runs Flash-Next with the conductor on. With Open WebUI’s own system prompt and 49 tools, 20 of 20 answers said so, against 0 of 20 on v0.8.0 (measured).

Smaller fixes.

The Flash-Next profiles load Laya, so the memory features work there: the remember, forget and recall intent took 20.6 ms on 3.05. A deleted chat’s sandbox stops at once; a restore brings the chat back, and the purge after 7 days or on ?forever=1 removes the container and the chat’s folders. A 404 from the community sync is a note, not an error.

36.8 0.8.1, a new box fetches every file

A new DGX Spark runs every 27B EXL3 profile and Flash-Next from public files, with no Hugging Face token (OPS-95). ops/use-profile.sh downloads what a 27B profile reads besides its weights before it switches: the K3 drafters and the K4 draft head, the BF16 drafters into train/, and the FP8 base checkpoint into the Hugging Face cache. It pins each download to the hub’s commit, checks each weight file’s sha256, checks the disk first, and continues an interrupted run; --dry-run prints every URL and size. The EXL3 profiles read the BF16 drafters from train/ on every box. A switch from setup mode removes the marker first and puts it back if the profile does not start. install.sh installs v0.8.1 by default, and --repo names the repository it clones. The engine’s modules are the bytes of 0.8.0. Section 3.3 has the steps for a new box.

Public weights.

The six EXL3 sizes of the 27B and their drafters are public on Hugging Face: 0xBakeer/TandemLLM-Qwen3.8-27B-EXL3-<bpw> for 2.5, 3.0, 3.5, 4.0, 4.5 and 6.0 bits a weight, and 0xBakeer/TandemLLM-Qwen3.8-27B-EXL3-drafters, in one collection. Each card has the size’s KL against BF16 and a speed table; KL runs from 0.1112 at 2.5 bits to 0.0018 at 6.0 (Table 9.2). The calibration repository stays private, and nothing that runs reads it.

36.9 0.8.0, Qwen3.8-Flash-Next and the Chat tab

A second model, Qwen3.8-Flash-Next, served from turboderp’s EXL3 packs through the same server, caches, ops scripts and setup guide as the 27B (Chapter 17). The 27B’s text does not change: its modules, flags and disk-tier fingerprint are as in 0.7.2, and every switch added for Flash-Next is off for it. On Flash-Next, greedy and seeded answers with speculation equal plain decoding token for token.

The Chat tab.

A chat in the dashboard backed by chatd, its own service behind the engine’s /v1/chat-app: runs that outlive the browser, tools in a sandbox per chat, the workspace panes, MCP servers, plan mode, the PWA. The Context panel reads the usage frame’s split by part and the cache peek; a chat on the NVMe is read back while the engine is idle (/v1/cache/prewarm). Found by the live checks and fixed: a restart’s restore that faulted the GPU on Flash-Next, the side lane on Flash-Next, its missing model card, a side slot’s chat that came back cold after a restart, a peek that answered 409 right after a turn, a prewarm that held a request for its whole read (it now stops between blocks), a chat waiting for a permission prompt that held one of chatd’s two run slots, and an iPhone test, where five screenshots counted by their base64 filled the window (319,136 tokens estimated against 16,980 real). The Chat tab carries a Beta pill in this release. On a phone the bottom bar has four slots with the rest behind More, and the composer shows every control.

Chat tab (beta): known limitations.

The Chat tab is in beta in this release, and a Beta pill beside it says so. We know of these limits and have not fixed them yet:

• The agent loop does not yet work the way Claude Code does (epic VIS-E10). Its prompts are not kept prefix-stable from step to step (SRV-375). Old tool results and old thinking stay in the context until a compaction (SRV-365). A compaction writes a general summary, not the task’s state (SRV-373). Every sandbox command asks for permission; there is no mode that runs them without asking (SRV-387). Nothing notices when the model repeats the same failing approach with different calls (SRV-367). render_page passes on Chromium’s console noise (SRV-369). Long tool output has no “see more” paging (SRV-371). Searches run in the main context, with no sub-agents (SRV-381).
• The engine reads a short Flash-Next chat again on its next turn when the answer ended inside a verified block (ENG-235).
• A request that arrives during a prewarm still waits 0.5 to 1 s (SRV-349).
• When memory reaches its ceiling, the engine drops the parked chat without writing it to the NVMe, so its next turn starts cold.
• Images only: there is no video input (VIS-97).
• The 5.05 Flash-Next pack does not load yet.
• Parallel decoding does not raise throughput over one request at a time: three requests together took 27.18 s against 27.03 s one after another, though the later ones start sooner.
• The phone layout’s keyboard behaviour is not verified in Safari on an iPhone.

Profiles and windows.

Five weight profiles, flashnext-2.05 to flashnext-6.05, and a second tile in the setup guide. Each pack has its largest window in the registry: 262,144 tokens for 2.05 and 3.05, 208,896 for 4.05 with the vision tower on; 5.05 does not load yet (its experts use 5 and 7 bits a weight), and 6.05 fits no window on one DGX Spark. 3.05 reads its n-gram table cold. A start with a longer --max-len is refused before the load.

Speculation.

The pack’s own MTP layer drafts, cut by the shared StairCut, with the lookup in the same tree. StairCut keeps its prices, the plain step’s cost and a drafted round’s cost per context class, takes the plain step where drafting does not pay, and captures missing graphs while the server is idle. One greedy request a length and its repeat, second round on a fresh server: 56.6 and 57.3 tok/s at 1k against 41.0 and 45.4 plain, 42.1 and 52.4 at 140k against 33.9 and 41.4; at 260,800 tokens the class takes plain turns (measured).

Caches and slots.

Every cache tier holds Flash-Next’s state on a 4,096-token grid. The KV pool, side slots and a joint verify pass work on it without changing any answer, and its profiles turn them on with three slots. Three requests arriving together take as long as one after the other (27.18 against 27.03 s), and the later first tokens come at 2.2 to 6.1 s instead of up to about 20 s (measured). The mixers are int8 with a scale a group of 128 weights: KL 0.0031 on code and 0.0188 on prose against the fp16 mixers.

Images and saved chats.

Flash-Next reads images: the pack’s vision tower runs on the EXL3 kernels, and its profiles turn it on (VIS-97). An image chat answers the same with the MTP drafter and without it, and the text after 140 image rows stays at KL 0.019 against transformers (measured). The disk tier now knows its files by their content (the fp4 fingerprint), not by their folder. Saved chats of 0.7.2 carry over in place, or from the old folder that QSE_DISK_ADOPT_FROM names; on the reference box Flash-Next kept its 34 saved chats through the switch.

36.10 What 0.7 added

0.7.0: the setup guide.

The dashboard’s #/setup walks through an install in eight steps, with a memory bar that compares the selection’s estimate with the box’s free memory. One formula in server/setup/estimate.py gives the numbers, and the page’s copy must pass the same cases. The models come from models/<id>/model.yaml. Wren, Qwen3.5-2B on the engine’s own forward, answers from the docs, the manual and the registry and can fill fields, but cannot apply, reset, load or unload anything. Finish runs an install job of fixed steps that brings back the old profile when its check fails; reset has five levels; install.sh --setup starts the dashboard and Wren alone on a box with no engine yet. Every POST under /v1/setup must be same-origin JSON. The text does not change: with Wren loaded on the narrator’s copy, three greedy prose requests gave the same 411 tokens as without it, at 148.6 against 147.5 tok/s (measured). The KV figure is corrected to 104.25 KiB a token.

0.7.1 and 0.7.2.

Wren is drawn as the DGX Spark box, with eight states and a still face for reduced motion. It answers a question in words again: a tool call may name only what the page can highlight, an invalid change gets one repair, and the last round sends no tools. On ten change requests and five questions it was right 15 times of 15 (measured). Neither release touches the engine’s modules.

36.11 What 0.6 added

0.6.0: one shared KV pool.

Each slot reserves addresses for up to 262,144 tokens and maps 2 MiB pages as rows are written (QSE_KV_POOL=on); pages past what a slot keeps go back when its request ends. Four slots of 262,144 tokens fit beside Laya and the narrator, with 58 to 63 GiB of MemAvailable after warm-up against 26 with the old 32k side windows, and the 4-request load test went from 28.4 to 31.5 tok/s on 2 slots to 34.2 to 35.6 on 4 (measured). A request that does not fit waits, says so in its stream and gets a 503 memory_wait_timeout only after the queue timeout. The logits, every KV row and the recurrent state match the static buffers bit for bit.

0.6.1 and 0.6.2.

When the pool runs short, the caches give memory back first, then the shadow reader, the narrator, idle side conversations and slot 0’s idle conversation; the models come back through the residency planner once memory stays free. An EXL3 prefill offers the GPU between its layers when another slot waits: in a test of three streamed chats beside a 37k-token prefill, the seconds without a decoded token went from 42 of 147 to 0 of 145 (measured). The dashboard’s memory split counts committed pages, and its Perf tab reads the 0.6 live feed again.

36.12 What 0.5 added

0.5.0: weight profiles.

The engine serves the NVFP4 set or an EXL3 quant of Qwen3.8-27B from 2.5 to 6.0 bits a weight, one at a time, through ops/use-profile.sh, which fetches missing weights, starts the profile, runs a smoke request and brings the old profile back when it fails. EXL3 linear layers have one kernel per bit width from 2 to 8 for 1 to 32 verify rows, and the trellis decode matches exllamav3 bit for bit; the drafters run in EXL3 at 3 bits. Against BF16 on 16,376 held-out positions, NVFP4 has a KL of 0.0582 and agrees on the top token 93.35 % of the time, EXL3 4.5 0.0217 and 95.78 % (measured). An EXL3 profile serves other weights, so its text differs from NVFP4’s; speculation stays exact against whichever weights run.

0.5.1.

The dashboard’s cache hit counts only the client’s own prompt tokens, and the live prefill rate only the rows the engine computed. The reference box runs four parallel requests.

36.13 What 0.4.8 adds

0.4.8 is 0.4.7 plus the 0.4.8-test builds of 2026-10-08; CHANGELOG.md has them with their live checks.

Parallel requests.

A side slot takes a request whose window leaves it at least 16,384 rows of output, with max_tokens cut to the room (0.4.7 compared the window with the whole served limit, so a chat past about 24,000 tokens never took the side slot and two chats took turns on slot 0). A side slot has a resident prefix of its own: the next turn of the conversation it holds resumes at the end anchor there, beside slot 0’s conversation; the router’s question in between leaves it in place. A new conversation beside one a side slot could never hold takes the idle side slot first, and a short one waits up to 20 s for a busy side slot before it sends the long conversation to the NVMe.

Caches.

The end alignment’s third pass serves a block the client moved from an earlier user message to its last one (Open WebUI’s retrieval block) from the rows where the model saw it, token-aligned on the bare bytes and only when the same bytes are in the last message; a message the user edited is served as sent. The router’s notes stay live by their message when the prompt changed before them. The RAM park keeps the end anchor, so a parked Open WebUI chat comes back at its end. A long prompt the end anchor does not serve says where the copy parted and what was refused, in the log and the journal.

Speed.

The speed guard’s short gate on the 0.4.8 engine code: 50.64 against release 0.3.0’s 49.75 tok/s (measured, one run a side, store off).

36.14 What 0.4.7 added

0.4.7 is 0.4.0 plus the test builds 0.4.1-test to 0.4.7-test; CHANGELOG.md has each one with its live checks.

Packages.

The system set (tandem-core, render-tools, eval-judge), four library packages (creative-coding, charts, voxel-worlds, explainer-video) and 34 everyday packages with everyday-core and reply-format. Every prose class offers the whole everyday set; the package question runs once per class (package_sticky: class) on an idle side slot. Measured against no package, 91 pairs judged by the served model: a mean of +1.19 of 5, indicative only; five everyday packages ship off. Everyday answers draft first and never stop on a form.

Requests.

Regenerate samples (temperature 0.7, top-p 0.95, top-k 20, a fresh seed; X-Tandem-Regenerate); every other answer stays greedy, and the ledger’s sampling column (schema 9) says which. Continue Response continues the partial answer. A memory note the store refuses is never shown, and a project note without a working directory lands at user scope.

Caches.

A long Open WebUI chat comes back from the NVMe after other chats ran (a 48k-token chat in 1.0 s, TTFT 2.3 s). The session spill is compressed without loss (1.71$\times$ on a real entry’s rows), the spill budget is 64 GB and the disk tier 16 GB, and the NVMe writes at most 200 GB a day. A conversation that ran on a side slot is carried into slot 0. Open WebUI chats own their NVMe entries, and a deploy that only registers a knob keeps the disk’s files.

Speed.

The speed guard’s short gate on the 0.4.7 engine code: 49.93 against release 0.3.0’s 50.38 tok/s (measured, one run a side, store off).

36.15 What 0.4.0 brought

The conductor.

The engine serves one model, TandemLLM, with a conductor in front (Part IV): profiles; a router that decides each new user message’s task class with a decision model and then Qwen itself; effort levels that are real caps on the reasoning; hidden notes with packages and skills, replayed at the same position; thinking budgets in every shape clients send and an answer reserve; side requests; the side lane; waiting streams with an estimate; status lines and the voice; the session monitor with its loop guards, tool checks and claim check; the tool-argument loop guard; the shadow reader; memory notes; session transcripts, turn snapshots and outcomes; session addresses. X-Tandem: raw turns all of it off for a request. Every client is the one user whose memory notes the engine keeps; X-Tandem-Memory: off keeps a request out of them. The testing packages name the engine as {tandem_url} and keep scratch files in {tandem_scratch}, out of the deliverable, put the deliverable before any test tooling and stop a test that fails the same way twice, and check a page with the kit’s own headless Chrome (laya_drive.py page) instead of a driver of the model’s own. The turn budget (S5) ends an agent turn that loops on its own test, sets up before it writes, or keeps a chat waiting. Asked who it is, the model names its model and TandemLLM. A prose request is not routed to a code class on a weak guess. A class’s core skill goes in on every turn of the class, and an unsure deliverable is built as picked and stated in one line instead of asked. Each Open WebUI chat is its own session (its chat id as a connection header, or its first turn without one), and a new folder never replaces the user’s files (the files-care package).

Requests.

A malformed request is a 400 that names the field; developer is the system prompt; usage.prompt_tokens counts what the client sent and the conductor’s tokens are prompt_tokens_details.hidden_tokens; a JSON answer has its thinking in reasoning_content, and so has every client but the Hermes agent under a conductor plan (no literal </think> in Open WebUI’s live view, none in opencode or an SDK’s answer); stop strings apply to the answer only; a request without tools never gets tool_calls; cached_tokens counts the client’s tokens below the point where the request resumed. A class’s packages fit the request’s tools, a reply-only instruction is chat, and the follow-up of an answer that thought resumes at its end. While the engine is away, a status responder answers the reverse proxy’s fallback with a page and a 503 that say what is going on.

Parallel requests.

--parallel N, off by default: side slots beside the engine, a packed draft and a joint verify of at most 32 rows for the decoding requests, admission by bytes; two requests at once 1.38$\times$ the total of one, four 1.42$\times$ (Section 14.13); a turn that continues slot 0’s conversation, judged on the client’s own tokens, waits for slot 0 and resumes in place, so an agent session keeps its cache with parallel requests on; the side slots’ bytes stay in the memory limits, and the router’s short questions and a client’s title tasks take an idle side slot, so a new user message resumes the conversation in place; Open WebUI’s live working directory moves from the tool block into a note. A request that shares only its client’s head with slot 0’s conversation takes the side slot, and a side slot reads the disk tier.

Caches.

The end anchor; the parked session; the engine’s memory limits and the memory gate; the disk tier on the NVMe; live conversations on the NVMe through a restart or an eviction; compaction as a continuation with a summary written ahead; images left out of long conversations in steps; a canonical skills list for opencode; a tokenised-prompt cache.

The dashboard.

The Conductor view with all its screens, the Parallel page, the hosts and the memory plan in the System view, the Backends and Setup views, filters by backend and user, and a phone layout for every view. TandemLLM’s own calls are left out of Usage unless asked, and the Live panel’s cached tokens never read above the prompt. It asks for no password: the admin routes are open until an admin password is set, so the engine belongs on a trusted network (Section 3.8).

The proxy.

tandem-dash: the dashboard in front of any OpenAI-compatible engine or cloud API, six backend kinds, routing by model id, fallbacks, one config database overlaid by TANDEM_* variables, encrypted keys, per-user keys, cost at each model’s price; vLLM’s and SGLang’s own metrics, the plugins inside them and tandem-dash launch; the host collector; the conductor in front of any backend, with the memory writer in the backend’s idle time.

The raw path runs 0.3.0’s arithmetic: every build of the series passed the identity gate against the one before it (tools/flagoff_identity.py, a fixed scenario of fixed shapes, logits and state bit for bit). With the same tree shapes it writes 0.3.0’s text, with one exception: the raw path no longer closes a reasoning block that repeats itself (0.3.0’s stall close, on by default); under a conductor plan the session monitor’s reasoning-loop task (D1) does that job. The length router sizes blocks from timings, so two runs of either version can part at a near tie (Section 12.3). The last benchmark row recorded in the series, with --no-learn-cost and 50 requests, read 50.09 tok/s (measured); the paper reports 49.89. The release’s speed check, 0.3.0 and 0.4.0 on the same box in the same hour with one run each, read 50.50 and 51.38 tok/s (measured, release/speed-guard.json). The upgrade notes are in Chapter 3.

Not verified in this release.

OpenAI’s adapter with a paid call (it is tested on frames written from OpenAI’s documented shapes); the conductor’s second requests on a real vLLM or SGLang (tested on a fake backend); the plugins’ check against running servers; the per-process GPU memory on a discrete GPU; the docker image (the wheel it installs is built and tested).

36.16 Measured in the 0.4.0 series

Table 36.2 lists the measurements this manual cites from the series, all on the published weights on one DGX Spark.

what	value
side lane: 10 short requests fired into a 5,000-token turn	9 served at 1.2 s median; turn +7.4 s; the turn’s state comes back bit for bit, its text only while the tree shapes match
compaction of a 57,008-token request	115.5 s as sent, 40.7 s as a continuation, 0.07 s prepared
session spill: a 41,587-token conversation	written in 1.03 s (5.20 GB), read back in 0.94 s
disk tier: a 15,360-token head	restored in 334 ms; TTFT 23.1 s down to 2.7 s
the voice, Qwen3.5-2B	decode $-1.90$ % (63.47 to 62.26 tok/s)
the shadow reader, Qwen3.5-0.8B, 16,350-token prompt	decode $-2.05$ % (36.02 to 35.28 tok/s)
memory notes on and off	99.7 against 99.6 ms a block
the proxy against a fake backend	+0.33 ms to TTFT at the median
the host collector, one sample with a process scan	7.46 ms of CPU