38Changelog summary

CHANGELOG.md in the repository lists every release with its measurements. This chapter summarises it. Every release is held to the one before it on the raw path by the identity gate (the same logits and state, bit for bit, on a fixed scenario), except where an entry says otherwise, and each one’s speed is checked on the benchmark row against the release before it, in the same hour. The gate fixes the arithmetic, not the tree shapes the length router picks from timings, so the served text can part at a near tie between two runs of either release (Section 12.3).

release	date	what it brought
0.1.0-rc1 to rc4	2026-09-18 to 09-25	the engine, the length router, release of the losing drafter, serving defaults for long work; rc4 is the first release measured in two store views
0.1.0-rc5 to rc7	09-26 to 09-27	sampled requests on the draft tree, the in-process block A/B, tool calls read in three forms and streamed as argument deltas
0.1.0-rc8 to rc10	09-27 to 09-28	the resident prefix, live activity on the dashboard, the FP8 and balanced profiles, the first seams for other models
v0.2.0-staircut	09-29	StairCut, on in the served profile: the paper’s release
v0.2.1	09-29	the dashboard login off by default; no decode path changed
v0.3.0	09-30	image input; the text path unchanged
0.4.0	10-06	the conductor, memory notes, the caches for long agent sessions, parallel requests (off by default), and the dashboard as a standalone proxy in front of other engines and cloud APIs; not published, 0.4.7 replaces it
v0.4.7	10-08	0.4.0 plus the package library, Regenerate, long Open WebUI chats back from the NVMe, the compressed session spill and a daily NVMe write budget
v0.4.8	10-08	two Open WebUI chats beside each other (a side slot keeps its conversation), a chat whose retrieval block moved resumes at its end, the park keeps the end anchor, the router’s notes stay live by their message
0.5.0	10-09	weight profiles: NVFP4 or EXL3 at 2.5 to 6.0 bits a weight, one at a time, switched with ops/use-profile.sh
0.5.1	10-10	two dashboard numbers corrected; four parallel requests on the reference box
0.6.0	10-10	one shared KV pool that maps pages as a conversation grows; a request that does not fit waits instead of failing
0.6.1	10-10	the small models give memory back when the pool runs short; the memory split counts committed pages
0.6.2	10-10	decoding no longer stops while another slot prefills; the Perf tab’s live feed fixed
0.7.0	10-11	the setup guide at #/setup with Wren, its assistant; reset levels; setup mode
0.7.1	10-11	Wren drawn as the DGX Spark box; dashboard only
0.7.2	10-11	Wren answers a question in words again; setup guide only
0.8.0	10-13	a second model, Qwen3.8-Flash-Next, from turboderp’s EXL3 packs; the Chat tab with chatd; the cache peek and prewarm
0.8.1	10-13	a new box downloads every file a weight profile needs, with no token; the EXL3 weights public; the engine unchanged
0.8.2	10-13	community weight packs; permission modes in the Chat tab; standalone EXL3 packs of the 27B family; a chat no longer waits minutes for slot 0 while a side slot is free; the model says it is TandemLLM when asked
0.8.3	10-13	repository hygiene for the public release; no code change
0.9.0	10-14	a third model, LFM2.5-24B-A2B: six EXL3 sizes and BF16, the default 3.5 by a stated rule, the lookup drafting without a draft head; Wren beside it
0.9.1	10-14	web search and media playback in the Chat tab; the loop guard, the loop escape and memory notes on in the shipped conductor profiles; Setup Guide fixes; the 27B’s EXL3 profiles fetch only the base tensors they read
0.9.2	10-15	install fixes: the checksum step reads a download’s globs; after install.sh --setup the 27B’s EXL3 profiles take base-plain
0.10.0	10-15	a fourth model, Nemotron 3.5 Bolt 74B-A9B: seven EXL3 sizes, NVIDIA’s IQ2 GGUF and its LiLiCorr drafter; a 1 GiB limit for each Chat tab sandbox; attachments never inlined past the window
0.11.0	10-16	a fifth model, Nemotron 3.5 Super VL 120B-A12B: EXL3 packs at 4.0 and 2.0, NVIDIA’s NVFP4 release read as stored, its LiLiCorr drafter
0.11.1	10-16	chatd’s access to the sandbox’s folders follows no symlink; StairCut price files for Bolt’s EXL3 2.5 to 6.0; each weight pack’s maker in the setup guide; the install through the setup guide
0.11.2	10-16	chatd starts with the engine and the Chat tab says when it is down; install script and secrets fixes; sandbox containers scoped to their chatd; the preview opens the running app; Context panel figures; no horizontal overflow on a phone; a model picker with one row a model; Setup Guide steppers for chats and windows; Super VL window up to 1,048,576
0.11.3	10-17	Setup Guide hardening: tandem doctor, a check before the restart, Retry and Copy diagnostics, the restart card, removing weights; Super VL reads images; a cloud model’s figures and cost; slot affinity
0.11.4	10-18	Agents mode in the Chat tab: a main agent with workers, the Task card, workers in the Context panel; a worker’s and its parent’s cache rows kept; file texts out of a chat’s requests; a memory refusal compacts; cloud runs take no run slot
0.12.0	10-19	the tools profile for API clients that offer tools; structured outputs with tools; C5, one mutation per turn; the secret echo mask; the raw path’s thinking in reasoning_content for plain API clients; tool-eval-bench results
0.12.1	10-20	the first community pack on Super VL, styles01’s EXL3 3.5 bpw, text only; a request with images on a side slot; redirects kept in the port preview; eleven outside pull requests; SECURITY.md and a threat model
0.12.2	10-20	a Chat-tab run’s tool results fit the room left and read_file reads by whole lines; the port preview streams; LFM2.5 adopts its 0.9.x to 0.11.1 conversations again
0.13.0	10-20	Ornith-1.5-122B on the Qwen3.5 MoE trunk, from our EXL3 4.0 pack, drafting with its MTP layer; the conductor’s third tool-eval-bench round; room for Ornith’s caches; chatd fixes for long sessions
0.14.0	10-22	Recursive’s Flint from our EXL3 4.0 and 6.0 packs and the BF16 checkpoint, drafting with the lookup; a resumed session keeps its end anchor; a re-rendered tool call served as written; prompt attention over the KV heads as they are; a lost CUDA context fails /health; tools/release.sh
0.14.1	10-22	a call’s argument names matched to the offered tool’s schema; a call to a tool not offered mapped to one that is; Flint answers directly with thinking off; Flint’s e4m3 KV cache, off by default; model families behind adapters

38.1 0.14.1, adapters

0.14.1 matches a tool call’s argument names to the offered tool’s schema before the call leaves the server, on both paths and in every tool-call form, so Flint’s read with path reaches opencode as filePath. A call to a tool the request did not offer goes to the offered tool of its kind; a search or a listing with no such tool becomes a quoted grep or ls for the client’s bash tool, and a write never becomes a shell command. Through opencode’s checks at Flint 4.0’s card sampling, 8 runs an arm, the checks passed went from 28 and 29 of 32 to 32 of 32 (measured).

With thinking off, Flint’s prompt gets one line before the conversation that asks it not to show its reasoning and to answer or call a tool: greedy, it opened its answer with reasoning on 0 of 10 hard asks instead of 7, and the median answer took 153 tokens instead of 582. FLINT_KV_FP8=1 keeps Flint’s full layers’ keys and values in e4m3; it changes the output, so it is off by default. Turn it on in the Setup Guide (the Tools step, Long-context cache (FP8) under Features) or with the line QWEN38_FLINT_KV_FP8=1 at the end of ~/TandemLLM/run/ops/serve.env and ~/TandemLLM/bin/tandem restart; Chapter 21 gives its cost and how to check it took.

Each model family now registers itself in its own package and declares how it is served: its tool-call form, its reasoning, where system messages may stand, the files others build on and its kernel launch options. Guards hold v0.14.0’s fingerprints, and shared code names no model (Chapter 16). The other models write the same text as in 0.14.0 and keep their saved conversations; Flint’s fingerprint moves once, so its saved prompt-cache entries from 0.14.0 are rebuilt once.

38.2 0.14.0, Flint

0.14.0 adds Recursive’s Flint from our EXL3 packs at 4.0 and 6.0 bits a weight and from the BF16 checkpoint (Chapter 21), on a new family, engine/models/flint: sliding-window layers on a ring, a learned sink per head, full layers without positional encoding, and a sigmoid router with a shared expert. Every profile drafts with the lookup, which StairCut cuts from a price file with a plain price for each context class. On a DGX Spark, one request, 512 greedy tokens, thinking off, one run each, the 4.0 pack decodes at 128.2 to 136.6 tok/s plain and up to 480.0 with the lookup, every drafted text the plain one. Flint writes tool calls in its own arg-key form: the conductor’s checks read them in that form, a string argument streams as it is written, and the note after a call that does not parse shows the model’s own form with an offered tool, never a placeholder.

The other models write the same text as in 0.13.0 and keep their saved conversations: Flint’s edits to shared files take the old paths unless Flint asks for the new ones, and SAME_ROWS names the new bytes as the old. A turn resumed from a session snapshot keeps its end anchor, end alignment serves a tool call the template wrote back differently as the model wrote it, and every prompt attention reads the KV heads as they are, where LFM2.5’s prompt chunks, Ornith’s and NemotronH’s MTP head had repeated them per query head. After a lost CUDA context /health fails and the watchdog restarts the engine. In the Chat tab, screenshots fold in blocks, plan mode keeps the chat’s effort word, and the run monitor steers a run that circles the same files, then stops it. tools/release.sh makes a release with one command.

38.3 0.13.0, Ornith-1.5-122B

0.13.0 adds Ornith AI’s Ornith-1.5-122B from our EXL3 pack at 4.0 bits a weight (Chapter 20). It is a Qwen3.5 mixture-of-experts model that reads images, on a new trunk, engine/models/qwen3_5_moe. The profile drafts with the model’s MTP layer and the lookup: on a DGX Spark, one request, 512 greedy tokens, thinking off, one run each, 37.2 to 38.4 tok/s plain and 85.3 to 124.8 drafted, every drafted text the plain one. The engine’s top token agrees with exllamav3’s own forward on the pack at 498 of 512 positions.

The other models write the same text as in 0.12.2. The router kernel’s new bytes moved the fingerprints of Flash-Next, LFM2.5 and NemotronH; at 512 experts it launches as before, SAME_ROWS names the new bytes as the old, and their first start adopts 0.12.2’s saved conversations once. The release also carries the response cache’s key on the resolved effort level, the tool-eval-bench batch tools, the deliverable keep-out in shadow, grep on one file’s path, read_file past 4 MiB and a grammar test that passes every run.

Ornith’s caches have room: the profile’s limits are 86 and 90 GiB with a 2 GiB pin, where the first ceiling of 74 refused every end anchor and store snapshot. A refused end alignment serves the part it aligned instead of reading the turn again. In the Chat tab, writes and long commands stay whole in later requests until a fold point, plan mode keeps the request’s head, and a run stopped at its call cap leaves the chat idle. Every test file now runs its tests as a script, and a file that runs none fails.

38.4 0.12.2, large reads and a streaming preview

0.12.2 fixes two Chat-tab bugs that users reported, issues #18 and #86, and the adoption of LFM2.5’s older saved conversations. A run that reads a large file now ends with an answer instead of a request over the window: one step’s tool results share the room the model has left, read_file stops at whole lines and names the offset to read on from, and near the window chatd counts the request on our engine before it sends it (Chapter 29). The port preview passes an app’s Server-Sent Events and long-polls on as they arrive, and a browser that leaves ends the relay in the sandbox.

No model, card or profile changed, and every profile writes the same text as in 0.12.1. read_file’s schema changes, so each chat’s first request after the update reads its prompt again. LFM2.5 adopts the saved conversations of 0.9.x to 0.11.1 again; a box that ran 0.11.2 to 0.12.1 already lost them.

38.5 0.12.1, a community pack on Super VL

0.12.1 serves the first community pack on Nemotron 3.5 Super VL, styles01’s EXL3 pack at 3.5 bits a weight (pull request #63, Section 7.4). It is served with our 4.0 profile’s limits, LiLiCorr and the lookup, text only, since the pack holds no image tower. NemotronH now reads a stored head when the config says the head is tied, as this pack’s remapped config does. On a DGX Spark, one request, 512 greedy tokens, thinking off, one run each: 25.9 tok/s plain, and 33.2 prose, 53.9 code, 37.4 chat with LiLiCorr and the lookup; the top token agrees with exllamav3’s own forward on the pack at 510 of 512 positions.

A request with images now takes a side slot (Section 14.13); QWEN38_SIDE_IMAGES=off keeps it on slot 0. The Chat tab’s port preview keeps an app’s redirects inside the preview. Pull requests #78 to #85 by joeynyc and #34 and #43 by AgentSparko fix chatd’s attachments and port proxy, the Setup Guide’s remove and its timers, the GGUF reader’s size checks and a long thin image’s patch grid. Every profile writes the same text as in 0.12.0. The NemotronH profiles’ fingerprint changes, so the saved conversations of Bolt, Lightning and Super VL from 0.12.0 read their prompt again; the other models’ carry over.

38.6 0.12.0, tool clients

0.12.0 serves API clients that offer tools through a profile of their own. An OpenAI SDK, curl or an unnamed client that sends tools and names no profile gets tools (Section 22.3): the prompt as the client sent it, token for token the raw path’s, with the loop guards and the tool-call checks acting. A request with tools can ask for a JSON answer, and the model’s first answer token picks a tool call or the document. An answer that makes two or more changes outside the conversation in one turn is written again one call at a time (C5), and a key the model read in a tool result is masked where the answer repeats it. On the raw path a plain API client gets the thinking in reasoning_content.

On tool-eval-bench’s 69 scenarios, on one DGX Spark with Qwen3.8-27B in the EXL3 6.0 pack, the raw path scored 92 with thinking off and 95 to 97 with it on, and the tools profile 95. Two runs of the same prompt scored 95 and 97, so rows within 2 points are the same. Profiles that add packages took twice the time and scored no higher. In the bench’s hard mode, 28 scenarios, the conductor scored 91 against 84 for raw, one run a side. docs/tool-eval-bench.md has every row and its command. No change touches the decoding of a request without tools and without a JSON format; the served text against 0.11.4 was not measured.

38.7 0.11.4, Agents mode

0.11.4 adds Agents mode to the Chat tab (Section 29.5). With the chat setting agents on, the main agent plans and reads, and gives every file write and every multi-step build or fix to a worker through the task tool. A worker is a full run on a hidden conversation of its own that sees the task, its brief and the user’s request, and nothing else of the chat; its report comes back with chatd’s check of what it changed and ran. A Task card shows each worker’s run, report and transcript, and the Context panel shows each running worker apart, with its live prefill and decode figures. Stop, a queued message and Continue reach the worker. Agents mode is off by default.

The engine keeps a worker’s rows and its parent’s while they take turns. A request with X-Tandem-Worker or X-Tandem-Parent-Session is never a guest; for 600 s after a worker’s last request its parent’s rows go to the NVMe rather than away, a returning parent tries slot 0 first, and the park and the NVMe spill keep a conversation they used to drop. A request with images resumes at the end anchor (QWEN38_MM_END), and the live row has a prefill rate from its first mark. These restores copy rows the engine computed before and compute none; the tests hold every resume’s logits to a cold prefill’s within $10^{-4}$. The identity gate was not run against 0.11.3.

In a chat’s own requests, a written file’s text goes to the model as a stub from the next request on, a long read stays whole until a compaction, and a compaction cuts to under half the limit that set it off. A memory refusal compacts the chat and the run goes on, and a run on a cloud model takes no run slot. To upgrade, run the installer again, then tandem restart and tandem chat restart; chatd’s store moves to v8 in place.

38.8 0.11.3, the Setup Guide hardened

0.11.3 makes a Setup Guide install safe to retry. tandem doctor checks an install and repairs what is safe (file modes against ops/executables.txt, the run/ops copies, stale pid files, the secrets file’s mode); it runs before every tandem start and in the install’s check step, and as tandem doctor [--check] [--json]. A step before the restart checks the scripts, serve.env, the weights, the memory and the port, and a failure there stops nothing. The scripts call each other through bash, never through the execute bit. A failed step says what broke in plain words, with Retry from that step and Copy diagnostics. A restart is one card with the job’s step and the load’s phases, served by the tree’s own status responder while the engine is away; a finished install has its own screen with Start chatting. The weights step lists the weights on the box and removes a model’s weights, never the running profile’s or a shared folder. tools/release\_gate\_setup.sh installs a candidate twice in scratch folders beside the live engine and drives the whole flow in a phone-sized browser.

Super VL reads images through its C-RADIO tower. A cloud model’s turns in the Chat tab have their speed, the first token’s wait, the provider’s cache and the cost, and Performance and Usage keep the engine’s requests apart from relayed ones. A conversation stays on the side slot that holds its rows, the narrator is off by default, the decider’s residency claim is 3.0 GiB, and a chat sandbox has 4 GiB of memory. Requests on the raw path write the same text as in 0.11.2; under the conductor a turn can get packages it did not get before, and its text changes with them. To upgrade, run the installer again, then tandem restart and tandem chat restart; the conductor’s migrations run at the engine’s first start.

38.9 0.11.2, chatd with the engine

0.11.2 starts chatd, the Chat tab’s service, with the engine (tandem start, the systemd unit, the Setup Guide) and stops it with tandem stop; the Chat tab names chatd when it is down instead of switching to the in-browser mock. The installer copies roles.sh, watchdog.sh and hold.sh, bin/tandem loads its config on an install without ops/active-profile, and the Setup Guide’s reset keeps the secrets file the engine reads. A second chatd on the same Docker no longer adopts another’s sandbox containers, and a sandbox container removed outside chatd starts again at the next tool call. The preview pane opens the running app, the Context panel shows a live run’s figures, and the Chat tab has no horizontal overflow on a phone. The model picker lists models, one row each, and DeepSeek V4 Flash and V4 Pro through the proxy can be picked. The Setup Guide sets the number of chats and both windows with steppers, and Nemotron 3.5 Super VL takes a window up to 1,048,576 tokens, checked to 249,983 prompt tokens. Update every box that runs the Chat tab: run the installer again, then tandem chat restart. Every model writes the same text as in 0.11.1.

38.10 0.11.1, the Chat tab’s sandbox folders

0.11.1 is a security fix: chatd reads and writes the chat’s workspace and the sandbox’s home without following a symlink, for uploads, the file tools, snapshots and the Open WebUI runner’s files. Update every box that runs the Chat tab, and restart chatd (ops/chatd.sh stop, then start). Bolt’s EXL3 packs 2.5 to 6.0 get their StairCut price files, so every pack drafts at least as fast as it decodes plain (Table 19.2). Two contributions: a pinned asset downloads at its revision, and NemotronH’s fingerprint covers its GGUF reader and file, so Bolt’s and Super VL’s saved conversations from 0.10.0 and 0.11.0 read their prompt again once. The setup guide names each weight pack’s maker, and the install goes through the setup guide. Every model writes the same text as in 0.11.0.

38.11 0.11.0, Nemotron 3.5 Super VL

0.11.0 adds a fifth model, NVIDIA’s Nemotron 3.5 Super VL 120B-A12B (Section 19.5). The other models’ text and speed do not change, and their saved conversations carry over.

Super VL.

Our EXL3 packs at 4.0 and 2.0 bits a weight and NVIDIA’s NVFP4 release, read as stored (NVFP4 experts, FP8 Mamba2 and attention projections and head), each with NVIDIA’s LiLiCorr drafter for Super VL beside the lookup. The 4.0 pack is the default: KL 0.049 against BF16 (NVIDIA’s NVFP4 0.041), 31.1 tok/s for one request (NVFP4 19.6), and with LiLiCorr and the lookup 53.6 tok/s on code and 101.0 on edit asks (the mean of 3 runs). Text only: the vision tower is not served yet.

38.12 0.10.0, Nemotron 3.5 Bolt

0.10.0 adds a fourth model, NVIDIA’s Nemotron 3.5 Bolt 74B-A9B (Chapter 19). The other models’ text and speed do not change, and their saved conversations carry over.

Bolt.

The NemotronH engine (Mamba2, attention without positional encoding, relu$^2$ experts behind a latent projection), our EXL3 packs at 2.0 to 6.0 bits a weight, NVIDIA’s IQ2 GGUF read as stored, and NVIDIA’s LiLiCorr drafter beside the lookup. The 2.0 pack is the default: KL 0.170 against BF16 (the GGUF 0.259), 43.3 tok/s for one request, and with LiLiCorr and the lookup 65.7 tok/s on prose and 102.8 on code (the mean of 3 runs). A verify block’s EXL3 projections decode each trellis tile once for 16 rows, every output byte the one-row call’s. Text only: the image tower is not served yet.

Chat tab.

Each chat’s sandbox has 1 GiB of memory, and a call the kernel kills at the limit says so. An attachment is never inlined past the window: every attachment is saved under the chat’s /workspace/uploads/, and a text file past a quarter of the free window goes as one line with its path. A tool call no longer waits without end for its sandbox. POST /v1/cache/release gives back memory the engine holds idle.

Saved conversations.

Fingerprint v2 leaves out modules under engine/ that only another model imports, so the 27B’s code fingerprints stay those of 0.9.x; the first Flash-Next or LFM2.5 start adopts the files 0.9.x wrote, once.

38.13 0.9.2, install fixes

0.9.2 fixes two install bugs on a new DGX Spark. The engine, the conductor and the dashboard do not change.

The checksum step.

fetch_weights.py compared a download’s file globs to file names literally. A model whose list is all globs, Qwen3.5-0.8B for the shadow reader or Qwen3.5-2B for the narrator, checked no file and failed with “0 files checked: FAILED”, and Laya’s files under multilingual/ and typed-decisions/ went unchecked. The globs now select the files.

The EXL3 base after a setup install.

The installer wrote QWEN38_MODEL, the Hugging Face cache’s base folder, after the weight profile, so the 27B’s EXL3 profiles kept that empty folder instead of base-plain and use-profile.sh refused to switch. The installer’s own keys now come before the profile, an EXL3 profile takes base-plain when the folder holds no snapshot, and a standalone community pack reads its own folder. An install with HF_HOME writes the cache to serve.env as HF_HUB_CACHE. To upgrade, run the 0.9.2 installer again: it rewrites serve.env and the scripts beside it.

38.14 0.9.1, web search, media and the loop guard

0.9.1 changes the Chat tab, the conductor’s shipped profiles and the Setup Guide. The decode path does not change: under engine/ only the small models’ slot changed, so that a failed load is tried again.

Chat tab.

A built-in web_search tool with DuckDuckGo and SearXNG providers, set in the chat settings. Video, audio and PDF files open in a player or the browser’s PDF viewer, in the Files pane and the present card; any other binary file shows its size and a Download button. chatd streams a workspace file with byte ranges (206, 416, ETag) and no 2 MiB cap.

The loop guard.

The same-call guard (C3) is on in the shipped coding, chat-app, hermes, overnight and chat profiles. It counts only the current user turn, treats a tool result that took the client 10 s or more as a wait, and holds calls only on a request whose turn already ends with the repeat. After a repeated or near-repeated call the session samples with the model card’s settings (the checkpoint’s generation_config.json, a fresh seed) until it makes progress, then decodes greedy again; a client’s own sampling wins. Output changes on those turns only. --reasoning-effort none and off turn thinking off for every request that says nothing about thinking.

Memory notes.

The shipped chat, coding, hermes and chat-app profiles have memory: on. A one-time migration adds the switch to each stored profile without it; an explicit value stays. The memory card tells the model the notes are its memory between chats, a profile with memory off tells it so, and the Context panel shows a turn’s notes.

Fixes.

The Setup Guide’s install page keeps a finished job final and its whole log after a reload. The change list names the keys a model switch drops from run/setup.env. The fetch step downloads the models of the tools the selection turns on, and a failed shadow reader load is tried again after 300 s in an idle window. The 27B’s EXL3 profiles read their base from base-plain, fetched by HTTP range requests: 3.54 GB instead of 30.41, so a fresh exl3-6.0 downloads 36.48 GB, not 63.34. The dashboard’s service worker shows the cached shell when the reverse proxy answers 502, 503 or 504, and reloads once /health answers.

38.15 0.9.0, LFM2.5-24B-A2B

0.9.0 adds a third model, LiquidAI’s LFM2.5-24B-A2B, the first here that is not a Qwen (Chapter 18). The Qwen models’ text and speed do not change: the shared EXL3 kernels learned half-integer rates, and SAME_ROWS names their new bytes as the old ones, so the first Qwen start adopts its saved conversations once.

Profiles.

Liquid’s BF16 checkpoint and six EXL3 sizes of our own, 2.5 to 6.0 bits a weight, each at the model’s whole 131,072-token context with three slots. The decode step runs at 188.2 tok/s on 2.5, 153.5 on 3.5, 106.9 on 6.0 and 44.8 in BF16 (measured, 2026-10-14), with each pack’s own dense splits. The default is lfm25-exl3-3.5: the fastest profile in chat as it serves whose chat top-1 agreement with BF16 is at least 85 % (143.7 tok/s at 86.7 %; 3.0 is faster at 83.6 %). An exllamav3 -hq 4.0 was slower than 4.0 and worse than 4.5, and is not offered.

The lookup without a draft head.

A round with no lookup match is the plain step, priced as one; the lookup’s confidence carries its match length; every profile has its own StairCut price file. With the store off, edit and copy asks on 4.0 ran at 291.1 and 326.4 tok/s against 141.3 and 141.9 plain, every text equal.

The conductor.

On for LFM2.5 by default, with Laya and the narrator, through the model’s fit to the router: Laya alone answers the class question, a turn nobody classified gets nothing added, and a class sets the level and budget with only its core skill in the note. On 2.5, four short chat asks thought 2,234 reasoning tokens against 2,155 raw and 9,098 under the router as written; seven Chat-tab tasks run twice succeeded 14 of 14, as raw did, against 12 of 14 (Section 18.5.1).

Fixes.

A pack stored as one model.safetensors counts as complete in ops/use-profile.sh and the Setup Guide. Reading a slot’s kv.length commits nothing. Attention’s split buffers are kept per attention class. A suffix store of another tokenizer turns the store off instead of stopping the start. The dashboard’s disk tier default reads 16 GB, as the server’s. Wren loads its own Qwen3.5-2B beside LFM2.5 (20 s, 3.62 GiB).

38.16 0.8.3, repository hygiene

Repository hygiene for the public release: no private hostnames, home paths or personal notes in the files; no code change. The engine’s modules are the bytes of 0.8.2, and the text does not change. Comments, docs and tests no longer name a person who gave an instruction, and tests use 192.168.1.x for a LAN address and tandem.example.lan for a host behind a proxy. install.sh installs v0.8.3 by default.

38.17 0.8.2, community weights and permission modes

0.8.2 adds community weight packs, permission modes in the Chat tab and standalone EXL3 packs of the 27B family, and fixes two bugs found on v0.8.0 in Open WebUI on a phone. The engine’s modules are the bytes of 0.8.1.

Community weights.

One JSON file per pack under models/<model>/community/, checked by the registry against models/schema/community.schema.json and gathered in models/community-index.json. The setup guide lists the packs after our profiles as community/<id>, with search and five sort orders. It fetches the index from the public repository when its weights step opens or on Refresh, never on a timer, and falls back to its last copy, then to the one in the release; a 404 shows as the note “no published list yet”. ops/use-profile.sh community/<id> downloads a pack at its pinned commit, checks the hub’s sha256 and runs it with our drafters. An issue form and a check on pull requests come with it. The first entry is ThinkingCap Qwen3.8-27B Kearuga by 0xWhiteMage: EXL3 at 4.0 bits a weight, standalone, 39.6 tok/s prose, 95.9 code and 51.3 chat on 0.8.2 (measured). Chapter 7.

Permission modes.

Each chat in the Chat tab runs in one of three modes, and Auto is the new default: a call that stays inside the chat’s sandbox runs without a prompt, and one that leaves it still asks. Deny rules and plan mode’s read-only list hold in every mode. A tool card that ran without a prompt carries a badge, and every such call is logged as tool.auto_approved. “Always allow” on a chained bash command now saves a rule for every part not yet allowed; before, one chat saved cd 11 times and still got the prompt. Chapter 29.

Standalone EXL3 packs.

A complete BF16-origin mul1 EXL3 pack of the 27B family loads as --model, with no separate base checkpoint (pull request #1 by 0xWhiteMage). The path is text only, and speculative parallel serving still needs our drafters (Section 9.8). The 27B’s own profiles take the path they took before. engine/loader.py’s new bytes count as the old ones in the disk tier’s fingerprint (SAME_ROWS in engine/methods/manifest.py), so the first 27B start adopts v0.8.0’s saved files once, and the 27B’s code fingerprint stays 0.7.2’s.

The slot-0 wait.

A new Open WebUI chat waited 437 s for slot 0 behind another chat’s 496 s generation while both side slots were free. It shared only Open WebUI’s system prompt and tool list with an entry on the NVMe, and the wait for slot 0 had no limit. Now the spill wait needs the request’s own conversation, and every wait for slot 0 ends after 3 s (QWEN38_SIDE_SLOT0_WAIT_S) when a side slot the request fits in is free. /health parallel.max_running is the limit with its reason, not the high-water mark, and the Lanes view says why a request is queued. On the reference box a turn that slot 0’s chat had held moved to slot 2 after its 3.0 s wait, while another chat ran 80 s on slot 0 (measured).

Identity.

Asked who it was, Flash-Next named another assistant and its maker: the reference box served it with the conductor off, so the identity note never went in. The note now says the model is TandemLLM running the served model (Qwen3.8-Flash-Next or Qwen3.8-27B, from the pack’s model card) on the user’s own machine, for every client the conductor serves, and the reference box runs Flash-Next with the conductor on. With Open WebUI’s own system prompt and 49 tools, 20 of 20 answers said so, against 0 of 20 on v0.8.0 (measured).

Smaller fixes.

The Flash-Next profiles load Laya, so the memory features work there: the remember, forget and recall intent took 20.6 ms on 3.05. A deleted chat’s sandbox stops at once; a restore brings the chat back, and the purge after 7 days or on ?forever=1 removes the container and the chat’s folders. A 404 from the community sync is a note, not an error.

38.18 0.8.1, a new box fetches every file

A new DGX Spark runs every 27B EXL3 profile and Flash-Next from public files, with no Hugging Face token (OPS-95). ops/use-profile.sh downloads what a 27B profile reads besides its weights before it switches: the K3 drafters and the K4 draft head, the BF16 drafters into train/, and the FP8 base checkpoint into the Hugging Face cache. It pins each download to the hub’s commit, checks each weight file’s sha256, checks the disk first, and continues an interrupted run; --dry-run prints every URL and size. The EXL3 profiles read the BF16 drafters from train/ on every box. A switch from setup mode removes the marker first and puts it back if the profile does not start. install.sh installs v0.8.1 by default, and --repo names the repository it clones. The engine’s modules are the bytes of 0.8.0. Section 3.3 has the steps for a new box.

Public weights.

The six EXL3 sizes of the 27B and their drafters are public on Hugging Face: 0xBakeer/TandemLLM-Qwen3.8-27B-EXL3-<bpw> for 2.5, 3.0, 3.5, 4.0, 4.5 and 6.0 bits a weight, and 0xBakeer/TandemLLM-Qwen3.8-27B-EXL3-drafters, in one collection. Each card has the size’s KL against BF16 and a speed table; KL runs from 0.1112 at 2.5 bits to 0.0018 at 6.0 (Table 9.2). The calibration repository stays private, and nothing that runs reads it.

38.19 0.8.0, Qwen3.8-Flash-Next and the Chat tab

A second model, Qwen3.8-Flash-Next, served from turboderp’s EXL3 packs through the same server, caches, ops scripts and setup guide as the 27B (Chapter 17). The 27B’s text does not change: its modules, flags and disk-tier fingerprint are as in 0.7.2, and every switch added for Flash-Next is off for it. On Flash-Next, greedy and seeded answers with speculation equal plain decoding token for token.

The Chat tab.

A chat in the dashboard backed by chatd, its own service behind the engine’s /v1/chat-app: runs that outlive the browser, tools in a sandbox per chat, the workspace panes, MCP servers, plan mode, the PWA. The Context panel reads the usage frame’s split by part and the cache peek; a chat on the NVMe is read back while the engine is idle (/v1/cache/prewarm). Found by the live checks and fixed: a restart’s restore that faulted the GPU on Flash-Next, the side lane on Flash-Next, its missing model card, a side slot’s chat that came back cold after a restart, a peek that answered 409 right after a turn, a prewarm that held a request for its whole read (it now stops between blocks), a chat waiting for a permission prompt that held one of chatd’s two run slots, and an iPhone test, where five screenshots counted by their base64 filled the window (319,136 tokens estimated against 16,980 real). The Chat tab carries a Beta pill in this release. On a phone the bottom bar has four slots with the rest behind More, and the composer shows every control.

Chat tab (beta): known limitations.

The Chat tab is in beta in this release, and a Beta pill beside it says so. We know of these limits and have not fixed them yet:

• The agent loop does not yet work the way Claude Code does (epic VIS-E10). Its prompts are not kept prefix-stable from step to step (SRV-375). Old tool results and old thinking stay in the context until a compaction (SRV-365). A compaction writes a general summary, not the task’s state (SRV-373). Every sandbox command asks for permission; there is no mode that runs them without asking (SRV-387). Nothing notices when the model repeats the same failing approach with different calls (SRV-367). render_page passes on Chromium’s console noise (SRV-369). Long tool output has no “see more” paging (SRV-371). Searches run in the main context, with no sub-agents (SRV-381).
• The engine reads a short Flash-Next chat again on its next turn when the answer ended inside a verified block (ENG-235).
• A request that arrives during a prewarm still waits 0.5 to 1 s (SRV-349).
• When memory reaches its ceiling, the engine drops the parked chat without writing it to the NVMe, so its next turn starts cold.
• Images only: there is no video input (VIS-97).
• The 5.05 Flash-Next pack does not load yet.
• Parallel decoding does not raise throughput over one request at a time: three requests together took 27.18 s against 27.03 s one after another, though the later ones start sooner.
• The phone layout’s keyboard behaviour is not verified in Safari on an iPhone.

Profiles and windows.

Five weight profiles, flashnext-2.05 to flashnext-6.05, and a second tile in the setup guide. Each pack has its largest window in the registry: 262,144 tokens for 2.05 and 3.05, 208,896 for 4.05 with the vision tower on; 5.05 does not load yet (its experts use 5 and 7 bits a weight), and 6.05 fits no window on one DGX Spark. 3.05 reads its n-gram table cold. A start with a longer --max-len is refused before the load.

Speculation.

The pack’s own MTP layer drafts, cut by the shared StairCut, with the lookup in the same tree. StairCut keeps its prices, the plain step’s cost and a drafted round’s cost per context class, takes the plain step where drafting does not pay, and captures missing graphs while the server is idle. One greedy request a length and its repeat, second round on a fresh server: 56.6 and 57.3 tok/s at 1k against 41.0 and 45.4 plain, 42.1 and 52.4 at 140k against 33.9 and 41.4; at 260,800 tokens the class takes plain turns (measured).

Caches and slots.

Every cache tier holds Flash-Next’s state on a 4,096-token grid. The KV pool, side slots and a joint verify pass work on it without changing any answer, and its profiles turn them on with three slots. Three requests arriving together take as long as one after the other (27.18 against 27.03 s), and the later first tokens come at 2.2 to 6.1 s instead of up to about 20 s (measured). The mixers are int8 with a scale a group of 128 weights: KL 0.0031 on code and 0.0188 on prose against the fp16 mixers.

Images and saved chats.

Flash-Next reads images: the pack’s vision tower runs on the EXL3 kernels, and its profiles turn it on (VIS-97). An image chat answers the same with the MTP drafter and without it, and the text after 140 image rows stays at KL 0.019 against transformers (measured). The disk tier now knows its files by their content (the fp4 fingerprint), not by their folder. Saved chats of 0.7.2 carry over in place, or from the old folder that QSE_DISK_ADOPT_FROM names; on the reference box Flash-Next kept its 34 saved chats through the switch.

38.20 What 0.7 added

0.7.0: the setup guide.

The dashboard’s #/setup walks through an install in eight steps, with a memory bar that compares the selection’s estimate with the box’s free memory. One formula in server/setup/estimate.py gives the numbers, and the page’s copy must pass the same cases. The models come from models/<id>/model.yaml. Wren, Qwen3.5-2B on the engine’s own forward, answers from the docs, the manual and the registry and can fill fields, but cannot apply, reset, load or unload anything. Finish runs an install job of fixed steps that brings back the old profile when its check fails; reset has five levels; install.sh --setup starts the dashboard and Wren alone on a box with no engine yet. Every POST under /v1/setup must be same-origin JSON. The text does not change: with Wren loaded on the narrator’s copy, three greedy prose requests gave the same 411 tokens as without it, at 148.6 against 147.5 tok/s (measured). The KV figure is corrected to 104.25 KiB a token.

0.7.1 and 0.7.2.

Wren is drawn as the DGX Spark box, with eight states and a still face for reduced motion. It answers a question in words again: a tool call may name only what the page can highlight, an invalid change gets one repair, and the last round sends no tools. On ten change requests and five questions it was right 15 times of 15 (measured). Neither release touches the engine’s modules.

38.21 What 0.6 added

0.6.0: one shared KV pool.

Each slot reserves addresses for up to 262,144 tokens and maps 2 MiB pages as rows are written (QSE_KV_POOL=on); pages past what a slot keeps go back when its request ends. Four slots of 262,144 tokens fit beside Laya and the narrator, with 58 to 63 GiB of MemAvailable after warm-up against 26 with the old 32k side windows, and the 4-request load test went from 28.4 to 31.5 tok/s on 2 slots to 34.2 to 35.6 on 4 (measured). A request that does not fit waits, says so in its stream and gets a 503 memory_wait_timeout only after the queue timeout. The logits, every KV row and the recurrent state match the static buffers bit for bit.

0.6.1 and 0.6.2.

When the pool runs short, the caches give memory back first, then the shadow reader, the narrator, idle side conversations and slot 0’s idle conversation; the models come back through the residency planner once memory stays free. An EXL3 prefill offers the GPU between its layers when another slot waits: in a test of three streamed chats beside a 37k-token prefill, the seconds without a decoded token went from 42 of 147 to 0 of 145 (measured). The dashboard’s memory split counts committed pages, and its Perf tab reads the 0.6 live feed again.

38.22 What 0.5 added

0.5.0: weight profiles.

The engine serves the NVFP4 set or an EXL3 quant of Qwen3.8-27B from 2.5 to 6.0 bits a weight, one at a time, through ops/use-profile.sh, which fetches missing weights, starts the profile, runs a smoke request and brings the old profile back when it fails. EXL3 linear layers have one kernel per bit width from 2 to 8 for 1 to 32 verify rows, and the trellis decode matches exllamav3 bit for bit; the drafters run in EXL3 at 3 bits. Against BF16 on 16,376 held-out positions, NVFP4 has a KL of 0.0582 and agrees on the top token 93.35 % of the time, EXL3 4.5 0.0217 and 95.78 % (measured). An EXL3 profile serves other weights, so its text differs from NVFP4’s; speculation stays exact against whichever weights run.

0.5.1.

The dashboard’s cache hit counts only the client’s own prompt tokens, and the live prefill rate only the rows the engine computed. The reference box runs four parallel requests.

38.23 What 0.4.8 adds

0.4.8 is 0.4.7 plus the 0.4.8-test builds of 2026-10-08; CHANGELOG.md has them with their live checks.

Parallel requests.

A side slot takes a request whose window leaves it at least 16,384 rows of output, with max_tokens cut to the room (0.4.7 compared the window with the whole served limit, so a chat past about 24,000 tokens never took the side slot and two chats took turns on slot 0). A side slot has a resident prefix of its own: the next turn of the conversation it holds resumes at the end anchor there, beside slot 0’s conversation; the router’s question in between leaves it in place. A new conversation beside one a side slot could never hold takes the idle side slot first, and a short one waits up to 20 s for a busy side slot before it sends the long conversation to the NVMe.

Caches.

The end alignment’s third pass serves a block the client moved from an earlier user message to its last one (Open WebUI’s retrieval block) from the rows where the model saw it, token-aligned on the bare bytes and only when the same bytes are in the last message; a message the user edited is served as sent. The router’s notes stay live by their message when the prompt changed before them. The RAM park keeps the end anchor, so a parked Open WebUI chat comes back at its end. A long prompt the end anchor does not serve says where the copy parted and what was refused, in the log and the journal.

Speed.

The speed guard’s short gate on the 0.4.8 engine code: 50.64 against release 0.3.0’s 49.75 tok/s (measured, one run a side, store off).

38.24 What 0.4.7 added

0.4.7 is 0.4.0 plus the test builds 0.4.1-test to 0.4.7-test; CHANGELOG.md has each one with its live checks.

Packages.

The system set (tandem-core, render-tools, eval-judge), four library packages (creative-coding, charts, voxel-worlds, explainer-video) and 34 everyday packages with everyday-core and reply-format. Every prose class offers the whole everyday set; the package question runs once per class (package_sticky: class) on an idle side slot. Measured against no package, 91 pairs judged by the served model: a mean of +1.19 of 5, indicative only; five everyday packages ship off. Everyday answers draft first and never stop on a form.

Requests.

Regenerate samples (temperature 0.7, top-p 0.95, top-k 20, a fresh seed; X-Tandem-Regenerate); every other answer stays greedy, and the ledger’s sampling column (schema 9) says which. Continue Response continues the partial answer. A memory note the store refuses is never shown, and a project note without a working directory lands at user scope.

Caches.

A long Open WebUI chat comes back from the NVMe after other chats ran (a 48k-token chat in 1.0 s, TTFT 2.3 s). The session spill is compressed without loss (1.71$\times$ on a real entry’s rows), the spill budget is 64 GB and the disk tier 16 GB, and the NVMe writes at most 200 GB a day. A conversation that ran on a side slot is carried into slot 0. Open WebUI chats own their NVMe entries, and a deploy that only registers a knob keeps the disk’s files.

Speed.

The speed guard’s short gate on the 0.4.7 engine code: 49.93 against release 0.3.0’s 50.38 tok/s (measured, one run a side, store off).

38.25 What 0.4.0 brought

The conductor.

The engine serves one model, TandemLLM, with a conductor in front (Part IV): profiles; a router that decides each new user message’s task class with a decision model and then Qwen itself; effort levels that are real caps on the reasoning; hidden notes with packages and skills, replayed at the same position; thinking budgets in every shape clients send and an answer reserve; side requests; the side lane; waiting streams with an estimate; status lines and the voice; the session monitor with its loop guards, tool checks and claim check; the tool-argument loop guard; the shadow reader; memory notes; session transcripts, turn snapshots and outcomes; session addresses. X-Tandem: raw turns all of it off for a request. Every client is the one user whose memory notes the engine keeps; X-Tandem-Memory: off keeps a request out of them. The testing packages name the engine as {tandem_url} and keep scratch files in {tandem_scratch}, out of the deliverable, put the deliverable before any test tooling and stop a test that fails the same way twice, and check a page with the kit’s own headless Chrome (laya_drive.py page) instead of a driver of the model’s own. The turn budget (S5) ends an agent turn that loops on its own test, sets up before it writes, or keeps a chat waiting. Asked who it is, the model names its model and TandemLLM. A prose request is not routed to a code class on a weak guess. A class’s core skill goes in on every turn of the class, and an unsure deliverable is built as picked and stated in one line instead of asked. Each Open WebUI chat is its own session (its chat id as a connection header, or its first turn without one), and a new folder never replaces the user’s files (the files-care package).

Requests.

A malformed request is a 400 that names the field; developer is the system prompt; usage.prompt_tokens counts what the client sent and the conductor’s tokens are prompt_tokens_details.hidden_tokens; a JSON answer has its thinking in reasoning_content, and so has every client but the Hermes agent under a conductor plan (no literal </think> in Open WebUI’s live view, none in opencode or an SDK’s answer); stop strings apply to the answer only; a request without tools never gets tool_calls; cached_tokens counts the client’s tokens below the point where the request resumed. A class’s packages fit the request’s tools, a reply-only instruction is chat, and the follow-up of an answer that thought resumes at its end. While the engine is away, a status responder answers the reverse proxy’s fallback with a page and a 503 that say what is going on.

Parallel requests.

--parallel N, off by default: side slots beside the engine, a packed draft and a joint verify of at most 32 rows for the decoding requests, admission by bytes; two requests at once 1.38$\times$ the total of one, four 1.42$\times$ (Section 14.13); a turn that continues slot 0’s conversation, judged on the client’s own tokens, waits for slot 0 and resumes in place, so an agent session keeps its cache with parallel requests on; the side slots’ bytes stay in the memory limits, and the router’s short questions and a client’s title tasks take an idle side slot, so a new user message resumes the conversation in place; Open WebUI’s live working directory moves from the tool block into a note. A request that shares only its client’s head with slot 0’s conversation takes the side slot, and a side slot reads the disk tier.

Caches.

The end anchor; the parked session; the engine’s memory limits and the memory gate; the disk tier on the NVMe; live conversations on the NVMe through a restart or an eviction; compaction as a continuation with a summary written ahead; images left out of long conversations in steps; a canonical skills list for opencode; a tokenised-prompt cache.

The dashboard.

The Conductor view with all its screens, the Parallel page, the hosts and the memory plan in the System view, the Backends and Setup views, filters by backend and user, and a phone layout for every view. TandemLLM’s own calls are left out of Usage unless asked, and the Live panel’s cached tokens never read above the prompt. It asks for no password: the admin routes are open until an admin password is set, so the engine belongs on a trusted network (Section 3.8).

The proxy.

tandem-dash: the dashboard in front of any OpenAI-compatible engine or cloud API, six backend kinds, routing by model id, fallbacks, one config database overlaid by TANDEM_* variables, encrypted keys, per-user keys, cost at each model’s price; vLLM’s and SGLang’s own metrics, the plugins inside them and tandem-dash launch; the host collector; the conductor in front of any backend, with the memory writer in the backend’s idle time.

The raw path runs 0.3.0’s arithmetic: every build of the series passed the identity gate against the one before it (tools/flagoff_identity.py, a fixed scenario of fixed shapes, logits and state bit for bit). With the same tree shapes it writes 0.3.0’s text, with one exception: the raw path no longer closes a reasoning block that repeats itself (0.3.0’s stall close, on by default); under a conductor plan the session monitor’s reasoning-loop task (D1) does that job. The length router sizes blocks from timings, so two runs of either version can part at a near tie (Section 12.3). The last benchmark row recorded in the series, with --no-learn-cost and 50 requests, read 50.09 tok/s (measured); the paper reports 49.89. The release’s speed check, 0.3.0 and 0.4.0 on the same box in the same hour with one run each, read 50.50 and 51.38 tok/s (measured, release/speed-guard.json). The upgrade notes are in Chapter 3.

Not verified in this release.

OpenAI’s adapter with a paid call (it is tested on frames written from OpenAI’s documented shapes); the conductor’s second requests on a real vLLM or SGLang (tested on a fake backend); the plugins’ check against running servers; the per-process GPU memory on a discrete GPU; the docker image (the wheel it installs is built and tested).

38.26 Measured in the 0.4.0 series

Table 38.2 lists the measurements this manual cites from the series, all on the published weights on one DGX Spark.

what	value
side lane: 10 short requests fired into a 5,000-token turn	9 served at 1.2 s median; turn +7.4 s; the turn’s state comes back bit for bit, its text only while the tree shapes match
compaction of a 57,008-token request	115.5 s as sent, 40.7 s as a continuation, 0.07 s prepared
session spill: a 41,587-token conversation	written in 1.03 s (5.20 GB), read back in 0.94 s
disk tier: a 15,360-token head	restored in 334 ms; TTFT 23.1 s down to 2.7 s
the voice, Qwen3.5-2B	decode $-1.90$ % (63.47 to 62.26 tok/s)
the shadow reader, Qwen3.5-0.8B, 16,350-token prompt	decode $-2.05$ % (36.02 to 35.28 tok/s)
memory notes on and off	99.7 against 99.6 ms a block
the proxy against a fake backend	+0.33 ms to TTFT at the median
the host collector, one sample with a process scan	7.46 ms of CPU