Changelog

Releases of TandemLLM. Speed numbers are the benchmark row serve-single-i256-o256-v1 (one request at a time, 256 prompt tokens, 256 generated, thinking off, greedy), mean tok/s with the lookup store off unless noted. docs/measurement.md explains the row. Every release produces the same text as the one before it, except where an entry says otherwise.

0.11.3, 2026-10-17

0.11.3 hardens the Setup Guide: tandem doctor checks an install and repairs what is safe, a check before the restart stops nothing when the new setup cannot start, the scripts call each other through bash, and a failed step says what broke, with Retry and Copy diagnostics. A restart is one card with the load's phases, a finished install has its own screen, and the Setup Guide removes a model's weights. Super VL reads images, a cloud model's turns in the Chat tab have their speed, cache and cost, and Performance and Usage keep the engine's requests apart from relayed ones. A conversation stays on the slot that holds its rows, the decider Laya fits beside Super VL, the narrator is off by default, and a chat sandbox has 4 GiB of memory. Requests on the raw path write the same text as in 0.11.2; under the conductor a turn can get packages it did not get before (SRV-534, SRV-536, SRV-552), and its text changes with them.

Install and the Setup Guide

• Our scripts call each other through bash, never through the execute bit: ops/use-profile.sh (start.sh, stop.sh), ops/watchdog.sh, install.sh's calls of bin/tandem and its systemd unit (ExecStart=/bin/bash DIR/bin/tandem start), and the documented cron lines. A Setup Guide install failed at "Restart the engine" after 1.4 s with ops/start.sh: Permission denied: a deploy had written start.sh with mode 644. tests/test_no_exec_bit.py fails on a new direct call in the scripts, the Python code's subprocess calls, the units and the documented command lines (OPS-167).
• The install page shows a job's true end within seconds of the engine's return (VIS-211). The job stream's catch-up gave the job snapshot the id of the log line before it, and the page, which had counted that id, dropped the snapshot as a repeat: with no job in hand it dropped every later step and the done event too, and a finished install showed "Step 8 of 8, Installing" and the spinner "Starting the install" for good. The snapshot carries no log line's id now, the page dedupes log lines only, and a page with no job, a stream that ends or an engine that answers again reads the job by id. After 30 s without the job the page says it cannot read it, with Try again and Refresh. A finished install has its own screen: " is ready", the weights, the tools, the parallel chats and their windows, how long it took, Start chatting and Back to the Setup Guide; the log and the steps fold under it. A tool that waits for memory reads "loads later, when memory frees up; nothing to do" (the planner's reason stays in the log). The dashboard's update banner no longer shows while a setup job runs or the engine is away, reads "The dashboard on the box was updated", and a waiting update loads at the finished screen's Start chatting. The install step no longer waits for the model's card, which cannot load during the restart.
• The status responder (ops/engine_status.py) belongs to its tree: GET /_tandem/status names its tree, status file, release and address, and ensure replaces a responder of another tree or an older release (found by its socket when it has no such route) and leaves any other program alone. During a Setup Guide restart on 2026-10-17 the page got a responder of another checkout, started the day before, which read that tree's status of 2026-10-15 ("TandemLLM is stopped: switching to Nemotron 3.5 Bolt") and answered the job's route with 404 ("This install and its log are no longer on the box"). A reason older than 6 h is no longer shown, and a start or stop that should long be over reads "not answering" (OPS-173).
• The responder carries a Setup Guide restart (OPS-175): the setup job as the server streams it (SSE or JSON, from the tree's run/setup-jobs), and GET /v1/setup/status with the engine's facts, the job's step n of N and the load's phase read from the engine log (weights, drafters, warm-up, verify graphs, the port). The server answers the same route while it runs, /health carries status_port, the read-only routes answer any origin, and the responder listens where the engine does. install.sh sets QSE_STATUS_PORT (the API port + 9), so every installed box has one. The fake engine's --fake-load-s N writes the load's phases over N seconds before it listens.
• A restart under a Setup Guide install is one card (VIS-209): "Restarting the engine with ", the job's step n of N, the load's phases (weights, drafters, warm-up, verify graphs, the port, with the weights' last progress line), the time taken and about how long is left, and a bar. The card asks /v1/setup/status every 2 s, of the engine, else of the status responder through the reverse proxy or on its own port (status_port, remembered from /health); with nothing to ask it counts the time against a typical start, and far past it says so with a refresh. The job this browser follows is kept in localStorage, so a refresh, a page the service worker served while the engine was away, or a view that went offline goes back to the Setup Guide's job and its card; the pill reads "restarting" instead of "offline", and the restart and offline banners, the update banner and the setup page's route errors stay away. When the engine answers, the card goes and the job's end shows (VIS-211).
• tandem doctor checks an install and repairs what is safe (OPS-171): file modes against ops/executables.txt (the release's executable files, kept equal to git's), the run/ops copies and bin/tandem, stale pid files of the engine and chatd, a hold's pause left behind, the setup marker's address, and the secrets file's mode (600). It reports what it does not change: a run/ops copy that differs from the checkout, a serve.env that does not load and its line, an interpreter that does not run, a secrets file where the engine does not read it, too little disk for a download. It runs before every tandem start and in every Setup Guide install's check step (the job keeps its report as doctor), and as tandem doctor [--check] [--json] or python -m server.setup.doctor; each run is logged in logs/doctor.log.
• A Setup Guide install checks what the restart needs before anything stops (OPS-169): a step of its own, "Check the new setup before the restart", between the download and the configuration. The scripts the switch runs are there and parse, serve.env with the new profile and the overlay the job will write sources under set -u as start.sh sources it, the weights are complete (the hub manifest's sizes too), MemAvailable and what the running engine gives back cover the profile's need (MEM_NEED_GB, as use-profile.sh), the new port is free, no box lock or hold, and the engine's and chatd's interpreters run. A failure ends the job there, says what is wrong in plain words and "Nothing was stopped or changed"; the running setup keeps running. Before, a start script without its execute bit was found only after the engine had stopped.
• A failed Setup Guide step says what broke in plain words (VIS-213): "the script start.sh could not be run (Permission denied: it lost its execute bit). tandem doctor has repaired its file mode; press Retry." instead of "the new setup did not start (ops/use-profile.sh exit 1)"; a hold, too little memory, an engine that did not answer in time (with its log's last line), a download that stopped and a time-out have their own sentences. The failed and rolled-back screens show the step it stopped at, what tandem doctor repaired, Retry from that step and Copy diagnostics. POST /v1/setup/jobs/{id}/retry starts a new job from the failed one's selection (retry_of; the check runs again, a finished download is not repeated); GET /v1/setup/jobs/{id}/diagnostics gives the job as text (versions, steps, tandem doctor, the selection, the last 200 log lines, no secret), which the page copies, or shows to select where the browser does not let it copy. The check step's change count shows once.
• The Setup Guide removes a model's weights (SRV-560, VIS-215). The weights step lists the weights on the box with their sizes (GET /v1/setup/weights; the running one marked, the shared drafters and base listed apart), and Remove asks first with the GB it frees and the line that downloads them again. POST /v1/setup/remove never removes the running profile, a folder a kept profile shares, or a folder outside the home folder and the Hugging Face cache, holding the installation or what something else reads; the review step can remove the old weights once the new setup answers (remove_after). Each removal is a line in run/weights-removed.txt with the repository at its revision and the fetch command. A download that does not fit names the GB too few and the weights that could be removed.
• A release gate for the Setup Guide (OPS-177), mandatory before every release (docs/operations.md, "Release checklist"): tools/release_gate_setup.sh installs the candidate twice on the box with install.sh --setup into scratch folders, beside the live engine and without touching it (spare ports, no chatd, the CPU fake engine with a load of 20 s, a few KB of stand-in weights, a HOME of its own, an interpreter copy the live box's ops/engines.sh does not count). Run A applies a profile through the Setup Guide's API while WebKit at 390 px follows the job in the Playwright container: the restart card during the restart, the finished screen with Start chatting within 10 s of the engine answering, screenshots; then the job, /health, a chat and a remove. Run B strips the execute bit from start.sh, stop.sh and use-profile.sh first and must still install. Its report and screenshots go to results/gate-setup/<utc>/.
• ops/start.sh asks /health every 2 s instead of every 10 s (600 s in all, as before), so a start, and an install's job with it, goes on within 2 s of the engine answering; a setup-mode start no longer counts as a model start in the status file's typical start, which made the next restart promise "any moment now". The dashboard's CSP admits the status responder's port (connect-src ... http://*:<QSE_STATUS_PORT>) and the responder answers a browser's preflight on its read-only routes, so a page with no reverse proxy in front reaches it (VIS-209, VIS-211).
• A Retry follows the new job from its first state: the page kept the failed job, final, over every event of the retry. A page that re-reads a finished job by id still reads its log to the end, and a job's last event before done carries its steps, the ones that did not run skipped (VIS-213). The release gate's browser part now fails the first try at the preflight, checks the failed screen and Copy diagnostics, presses Retry (no download), and removes a profile through the weights step's Remove button and its confirmation (OPS-177).

Chat tab

• Super VL's card says vision, so the Chat tab sends it the chat's images: uploads in the user message, the newest tool screenshot in its tool message (VIS-205).
• The Context panel lists the prompt's images as a part of their own, the engine's count, out of the conversation's share (VIS-205).
• A run that builds or changes a web page or app checks it in a browser before it ends. When the model's answer would end the run and web files changed since its start (the run's snapshot, so files written by bash count), chatd loads the page it presented, the project's running dev server or the changed page in the sandbox's headless Chromium at 1280 and 390 px with the laya kit's page check, as a call of its own (a verify card with a harness badge, the Now line "Verifying the app ..."). The model reads the console errors, failed loads, visible text, controls, the page's state hook, overflow and the screenshot's brightness and colours, and the 1280 px screenshot when it reads images, then fixes what is wrong or answers. A page that needs its dev server with none running gets the command instead. At most two checks a run, a second only after more changes; a sandbox failure counts in the run's failure streak (SRV-526). The answer ends with a "Checked / Not checked" block from the run's evidence: the checks, whether the model saw the screenshot, its own render_page, preview and kit calls, and files changed after the last check; with no sandbox, the step limit reached or a check that could not run, it says not verified and why. The chat setting verify (on by default, a switch in the composer's tools menu) turns it off. Before, a run that built a Three.js scene said it was done from a count of colours in a screenshot, and tested it only when asked (SRV-532).
• A cloud model's turn has its figures (VIS-207). chatd measures the stream itself: tok/s, the first token's wait and the thinking tokens live (a text chunk is a token on DeepSeek: 320 chunks for 321 tokens; a tool call's arguments count 3.5 characters a token), then after each call the provider's usage over the stream's span, DeepSeek's prompt_cache_hit_tokens as the cached tokens, and the proxy's price as cost_usd, shown in the line under the answer and the Context panel's usage. The proxy's live row of a relayed call counts no tokens, and chatd took its zeros over its own counts: a DeepSeek turn read 0 thinking through a minute of thinking and "not tracked for this model" for its speed. The Context panel says what is true for a cloud model: "no drafting (cloud)" for tokens per round and acceptance, "the provider decides" for the thinking budget, "cached by the provider" for the cache, the system prompt, tool schemas and skill index as chatd's estimate ("est."), the window from the model card (1,048,576 for DeepSeek V4) and the answer's room as a request asks for it (393,216, or what the window has left). The Now section shows the first token's wait for every model.
• A library text that downloads the laya kit (laya-playtest) gets the kit copied into the chat's files (/workspace/.tandem/) and a line that says so: the sandbox cannot reach TandemLLM's address, so the download failed. Its page check runs in the sandbox; the modes that ask Laya for each move do not (SRV-532).
• render_page reports the page's console only: Chromium's own pages (chrome://, "Slow network is detected"), its service errors (GCM registration) and its GL driver lines are dropped, the line keeps the message and its source without Chromium's process and clock, and WebGL runs through SwiftShader by name (--use-angle=swiftshader --enable-unsafe-swiftshader) instead of the deprecated automatic fallback and its notice. Every render of the 2026-10-16 pagoda chat said console: over eight lines of which none was the page's (SRV-538).
• A new chat's run may make 100 model calls before it stops for Continue (max_steps, 50 before); a chat's own value stays. The pagoda replay of 2026-10-17 wrote its 15 files in 22 calls and stopped at 50 while it fixed its first errors, before any check (SRV-544).
• The verify phase loads a page that loads module scripts from its own folder (<script type="module" src="https://github.com/0xBakeer/TandemLLM/raw/v0.11.3/js/main.js">) through the HTTP server that runs from that folder in the sandbox, not as file://, where Chromium refuses the modules (CORS) and the page stays blank; with no such server the check says so and how to serve the folder. Inline pages load as files as before (SRV-548).
• render_page on a port takes a workspace path below the server's folder as the path it serves (/workspace/_test.html on a server run from /workspace is /_test.html), and says when the server answered with an error ("the server answered HTTP 404 ...: the screenshot is its error page"); a 404 page read "no console errors" before (SRV-550).
• render_page runs the laya kit's page check (Chrome under DevTools, after the page has loaded) and reports console errors, uncaught exceptions and failed loads, and the visible text; Chromium's screenshot mode, which logs no failed load, stays as the fallback without the kit. A pagoda replay's module answered 404 and render_page said "no console errors" six times over the app's loading screen. An error that names a script of the page and a line (the kit now says where an exception was thrown) comes with that line from the chat's files, in render_page and in the verify phase's check: the same replay read "reading 'set'" at pond.js (62) thirteen times and edited main.js (SRV-554).
• A model call the engine ends with finish reason error is retried once as a new step, without the empty-turn nudge; a second fails the run saying the engine failed twice. A pagoda replay failed as "The model answered with nothing, twice" after two engine failures (SRV-556).
• A chat sandbox has 4 GiB of memory by default (QSE_SANDBOX_MEM_GB, 1 before; the running cap follows, 16 GiB for 4 sandboxes); a container made under another limit takes the current one at its next start. render_page and the laya kit start Chromium with --disable-dev-shm-usage and one renderer. Headless Chromium drawing a Three.js app in software went past 1 GiB and render_page was killed for memory on the app it checked (SRV-558).

Proxy and dashboard

• A streamed request with X-Tandem-Cost: 1 gets its cost as one SSE comment before data: [DONE] (: tandem-cost {"cost_micros": N, "backend": id}), the figure of its ledger row, the harness's second requests included; chatd sends the header. Without it the stream is the backend's, byte for byte (VIS-207).
• The card of a backend's model carries what the provider's model list says (VIS-207): DeepSeek's GET /models names the model, its window, its output limit, its input kinds and its effort levels, and the discovery every 10 minutes stores them. The picker shows "DeepSeek V4.1 Flash (cloud)" instead of the adapter's "DeepSeek V4 Flash"; the effort chip offers low, high and max for a DeepSeek model, auto being the provider's default (high), and the request carries reasoning_effort with that level (thinking off: none; the engine's medium and extra high go as high, a max picked on DeepSeek goes to the engine as extra high); deepseek-flash, whose input kinds name image, gets the chat's images (uploads, render_page screenshots) as image_url parts, deepseek-v4-pro none.
• A model id a later discovery finds for the first time is stored disabled, logged, and pushed once to the ntfy topic of the monitor's stall watchdog: nothing reaches the picker until it is turned on under Backends (VIS-207). deepseek-v4.1-flash-1017 was in DeepSeek's list for a while on 2026-10-17.
• The usage ledger records each relayed answer's system_fingerprint (schema 10, migrated in place), and Performance shows it in the request detail: deepseek-flash keeps its id while DeepSeek swaps the snapshot behind it (VIS-207).
• Performance and Usage keep the engine's requests and the relayed ones apart (VIS-207). Relayed requests were in the ledger with tokens, TTFT, tok/s and cost, but the views pooled them: an hour of a DeepSeek chat put the engine's decode p50 at 300 tok/s, its dots sat above the scatter's axis as "cold", the cache sources put the provider's cache hits under "cold", and the Live panel showed the request in flight as "warming up (0 of 50 tokens)". Recent requests now draws relayed requests as dots of their own on a log axis, the timing panel names the backend and the cost, the cache sources and the history count the engine's own requests with each relayed backend as a line of its own beside them in decode speed and TTFT, and Usage has a table of the engine against each relayed backend (requests, tokens, decode and TTFT p50, cost). The Live panel names a relayed request's backend and says its tokens are counted at its end; the Last 20 rows carry backend and show the backend's cache.

Models and engine

• Super VL reads images (ENG-365). The checkpoint's C-RADIO ViT-H/16 tower and projector (BF16 in the EXL3 packs and in NVIDIA's NVFP4 checkpoint, 1.57 GB of tensors) load beside the trunk: 1.60 GiB allocated on EXL3 4.0. An image is one tile at a dynamic resolution, 1,024 to 13,312 16-pixel patches, 256 to 3,328 prompt rows between <img> and </img> (1,000 for a 1,280 x 800 screenshot); the trunk has no positional encoding, so an image row is only an embedding over its placeholder. The processor is the checkpoint's, without remote code. Against the checkpoint's own processor and modeling code on three pictures (DGX Spark, 2026-10-17): pixel values and prompt ids identical, the tower's rows bit-identical to the reference's bf16 run (cosine 0.99973 to 0.99990 against its fp32 run, as the reference's own), greedy answers with our rows and with the reference's identical. The caches key image rows on the image's content. The NemotronH tier fingerprint counts vision.py, and engine.py changed: Bolt's, Lightning's and Super VL's saved conversations from 0.11.2 are not reused, their first request reads the prompt again.
• Rows forwarded after a prompt with images (a forced reasoning close, a hidden note) are plain rows: the serving forward handed them the image context and the request failed with "forward past the prompt ... with images attached" when the conductor closed a looping reasoning on a request with a screenshot (ENG-367).
• The three Super VL profiles set VISION=on; their memory need, ceiling and peak grow by 2 GiB (the tower and the 512 MB image cache). The setup guide has a Vision switch for Super VL with that figure. Bolt and Lightning stay text only.
• A turn that ends inside a verified block commits the block's rows through its end token only (SRV-530). The lookup drafts what followed the same tool call earlier in the prompt (</tool_call>, <|im_end|>, \n) and the walk accepted it; the loop committed those rows and returned at the end token, so the engine stood one or two rows past the committed tokens. A NemotronH state cannot take a committed row back: on Super VL 31 of 54 turns of a chat run kept no end anchor (end anchor not kept (ahead)), every one a tool call, and their next turn resumed at the last anchor line or read the conversation again. The commit now stops at the end token's row (or the budget's), where Qwen's cut back of the pending commit stopped: Qwen keeps the same state and end anchor bit for bit, NemotronH and LFM2.5 keep theirs, and the text is the same. end_cut counts these blocks in the resident prefix's report.

Parallel requests

• A conversation's next turn goes to the side slot that holds the longest share of its prompt, with or without an end anchor, and before slot 0 when slot 0 holds less of it (SRV-528). A side slot was its home only when it had kept an end anchor; without one a turn took the lowest free side slot, and once the 600 s carry guard of a slot below expired, a 45,683-token Super VL conversation moved from slot 2 to slot 1 and read 45,778 tokens again (80.9 s). The carry guard now orders new conversations only. A slot-0 conversation whose last request was cancelled inside its prompt, or that no request continued for QWEN38_SLOT0_KEEP_S (1,800 s), no longer sends new conversations to the side slots: a cancelled 1,000,000-token test prompt had kept the 1,048,576-token slot 0 idle all evening. A turn whose slot stays busy past QWEN38_SIDE_HOME_WAIT_S says it moved, in the log and the Lanes view; /health parallel counts hops and hop_tokens.

Conductor

• A turn whose class nobody decided keeps its class's packages under a model fit with fallback: lean (Super VL, Bolt, Lightning, LFM2.5): the profile's default class on a new session, the session's class after it, filtered by the fit's packages and rules as for a decided class. The lean fallback dropped every package, rule and core, so with Laya refused every Chat-tab turn on Super VL went out with no package at all; the Chat tab now gets code_edit's core. The effort stays the profile's default and no effort sentence goes in. A tool turn keeps the note of its user message, so a run's prompt prefix does not change between its steps (SRV-534).
• A router stored before FX6 gets the coding core on code_edit and debug by a one-time migration, unless a stored version gave them a core: under packages: core such a router gave a coding turn no package, decided or fallen back (SRV-534).
• design_ui and drawing have a core package, ui-design and creative-coding; a stored router gets them by a one-time migration where the class lists the package and no stored version gave it a core. Under packages: core (Super VL) a turn Laya decided as one of them got no package: "make a three.js page with a red cube" (design_ui, 0.43) went out with the reading note only (SRV-536).
• three_d's core is threejs-scene and coding; a stored router gets coding by a one-time migration unless a stored version had it there. Three pagoda replays on Super VL thought 6 to 11 minutes on their first step and drafted the files in the thinking; coding's "a plan of 3 to 8 lines, code in files only" reached code_edit and debug turns only (SRV-552).
• The narrator is off by default: the roles document's narrator.mode is off, and no shipped profile sets ui.narrator (coding and chat had live). The model does not load, the keeper leaves it, POST /v1/conductor/narrator/load answers 409 role_off, and the residency plan leaves out a role the roles document switches off, so the narrator's 6 GiB claim is gone. No model card turns the narrator on in the Setup Guide. A stored coding or chat profile with live, the role on in a stored roles document: off by a one-time migration, unless a stored version had another value. The narrator stays switchable: the role on and a profile's ui.narrator (SRV-534).
• The decider's residency claim is 3.0 GiB, measured with Laya in the engine (OPS-69), instead of the arbiter sidecar's 8.2. With 8.2 a new box refused Laya on Super VL at 3 slots of 262,144 (43.75 GiB available, 17.46 still to come to the engine's peak: 18.09 left, under the 21 of floor and margin); with 3.0, 23.29 are left. A stored residency document with 8.2 gets 3.0 by a one-time migration, unless a stored version had another value. While no Laya model is loaded because the planner refused it, the log says so once ([deciders] WARNING) and /health roles.laya.refused has the model, the reason and the numbers (SRV-534).

Packages

• threejs-scene: tone mapping and colour space, a day scene by default (a sky background with matching fog beyond the subject, hemisphere light about 1.5, a sun about 3 with shadows), a day/night switch that changes background, fog, lights and exposure together, saturated non-metal materials, foliage as clustered shapes, and a check of the screenshot against the request. Its skeleton had a near-black background and light strengths from before three.js r155, which render dim. The creative-coding Three.js skeleton gets the same light strengths and tone mapping (SRV-532).
• threejs-scene names the calls the model reached for that three.js does not have (Matrix4.makeRotationXYZ, Matrix4.rotateY, renderer.shadowMap.size, BufferGeometryUtils as a namespace) and what to use, the bloom chain with OutputPass last (without it the frame has no tone mapping), golden-hour and night light numbers, framing, and how to read a class's methods from the readable source of the same version. A pagoda replay spent 28 of its 50 calls on those four calls, grepping the minified build (SRV-542).
• The laya kit's page check reports each screenshot's look in numbers: brightness, the share of its most common colour, the colours over 1 % and saturation (SRV-532).

Upgrade: run the installer again, as for 0.11.2, then tandem restart and tandem chat restart. The conductor's migrations run at the engine's first start (migrations.json in the conductor's directory lists them), and the usage ledger moves to schema 10 in place. A chat sandbox made under the old 1 GiB limit takes 4 GiB at its next start.

0.11.2, 2026-10-16

0.11.2 starts chatd, the Chat tab's service, with the engine, and the Chat tab says when chatd is down instead of switching to the in-browser mock; update every box that runs the Chat tab. It fixes the install's scripts and secrets file, keeps a second chatd on the same Docker from taking another's sandboxes, makes the port preview open the app the model built, and reads the Context panel's live figures. The model picker lists models, one row each, and DeepSeek can be picked. The Setup Guide sets the number of chats and both windows with steppers, and Nemotron 3.5 Super VL takes a window up to 1,048,576 tokens, its checkpoint's own. Every model writes the same text as in 0.11.1.

Chat tab and install

• chatd starts with the engine (tandem start, ops/start.sh, the systemd unit, the Setup Guide) and stops with tandem stop; an engine restart leaves it running. The watchdog starts a chatd that died. tandem chat start|stop|restart|status|logs, a chatd line in tandem status, QSE_CHATD=off to opt out; the install's last message names the Chat tab (OPS-155).
• ops/chatd.sh runs under the install's venv and reads the same secrets file as the engine; a start that fails exits 1 with the log. ops/make-secrets.sh writes the file the engine reads, so the Setup Guide's generated admin password takes effect on an installed box. ops/chatd.service runs from ~/TandemLLM (OPS-153).
• The Chat tab says when chatd is not running, with the command, instead of switching to the in-browser mock, which now loads only for ?chat=mock; a key chatd refuses, an engine without chat routes and no answer are named too, and the tab checks again by itself (VIS-179). Reported by Aron (issue #3).
• bin/tandem loads its config on an install with no ops/active-profile, which is every plain and --setup install: under set -e the failed read ended tandem start, status and the rest with status 1 and no message, so the installer's own start failed. A Flash-Next profile with a cold table failed the same way. A config command that fails now names its file and line, and setup mode's tandem start no longer drops the page cache of every file under the working directory (OPS-159).
• The installer copies roles.sh, watchdog.sh and hold.sh into run/ops: every start on an installed box printed roles.sh: No such file or directory, and the documented watchdog line read the checkout's template serve.env. On an install the line is * * * * * ~/TandemLLM/run/ops/watchdog.sh (docs/operations.md); hold.sh there pauses the tree the watchdog and start.sh watch (OPS-161).
• The Setup Guide's reset never removes the secrets file the engine reads: it guarded ~/.qwen38-spark-engine/secrets.env only, and an install keeps it in DIR/state/secrets.env (QSE_SECRETS). The install's ledger in DIR/state/usage is kept below level 5 as well; the setup apply job reads $QSE_STATE_DIR/secrets.env in start.sh's order (OPS-163).
• render_page and the port preview dial ::1 and 127.0.0.1 by address, not through /etc/hosts: a dev server that listens on one of them and that curl reached answered "Connection refused", with no address and no port in the message (SRV-507). A Vite app's absolute module imports (/src/..., /@vite/client) and a style sheet's url(/...) are rewritten to the port's prefix, as the HTML was.
• The Context panel gives a live run's figures: the Now row keeps its tok/s between ticks, and a gap says why (between model calls, warming up, prefilling, queued, no drafter) where it read "not tracked for this model" for every one (VIS-181). The row of the call in flight wins over a finished one, a side slot's request has its own block counts, and a turn of several calls takes its first call's time to first token and the run's wall time, so "thinking, this turn" no longer reads 0 after a turn (SRV-509).
• The chat header's More menu opens downward from its button and fits the screen, so Pin, Rename, Export, Sandbox and Delete can be reached; it opened under the top bar (VIS-183).
• The preview pane opens the running app on its own: a port that starts listening opens in a Preview tab once, the plus menu's Preview opens the newest running server first, and a waiting pane names why, with the command and a Start button. port:N resolves to the server that listens when N does not, and a bundler project's source index.html resolves to its dev server. The pane serves module scripts and style sheets of the Vite dev server and of vite preview, and lets a page's fetch and hot-reload socket through. present reports the target, the server's answer for the page and its assets, and what the pane itself shows, and fails when the pane shows nothing (SRV-511).
• Chat tab design pass: no horizontal overflow from 320 px to 1440 px (a phone included), a quieter thread, cards, a figures line, and a tidier composer and panes (VIS-185).
• chatd adopts, reaps, stops, kills and removes only its own sandbox containers. Each chatd labels its containers with an instance id kept in its data dir (tandem.chat.instance); a container from before the label counts as its own only when the chat's workspace folder is in that data dir, and is created again with the label at its next start. Before, a second chatd on the same Docker (a test install with its own --data) adopted the first one's containers and removed them as containers of chats it did not know (SRV-524).
• A sandbox container removed or stopped outside chatd starts again at the next tool call, which says once that the sandbox restarted (files kept, processes gone). Before, chatd kept the box as running and every bash and render_page call answered exec failed (404). The same sandbox failure three calls in a row ends the run as interrupted with a notice, whatever the calls were; Continue goes on once the sandbox is back (SRV-526).
• The Continue card of an interrupted answer says why it stopped: its limit of model calls or the sandbox failing (the notice, in the card's place), the engine not answering, or chatd restarting. It said "The engine restarted" for every one, a run at its limit included. chatd records the reason in the answer's stats.interrupted; an answer from before has neutral words. The card hides while a Continue run writes into the same answer (VIS-197).
• The composer's model picker lists models, one row each: DeepSeek V4 Flash and V4 Pro through the proxy can be picked. It listed the conductor's profiles (each client's harness: opencode, Open WebUI, Hermes, A/B tests, header-only ones) as the only buttons, so a model without profiles was a heading, and both DeepSeek models were named by their backend, DeepSeek (cloud). A backend's model is named by its provider name (name in the proxy's /v1/models card, with cloud), else its id; the line under it leaves out what the backend does not report. The Chat tab runs under its own profile, chat-app, by its client kind: chatd sends no X-Tandem-Profile, a settings patch stores no profile, and chatd's store v7 clears the profile stored in a chat or preset. A preset picks a model instead of a profile. A chat with a DeepSeek model failed at its first call: chatd sent the window minus the prompt as max_tokens, and DeepSeek refuses more than 393,216. A backend's model now gets a max_tokens only from its output limit (max_output in the proxy's card, 393,216 for DeepSeek), none when the limit is unknown (VIS-203).

Setup Guide

• Parallel chats is a stepper: minus, a number field and plus (VIS-199). A model card's range is how far the model was measured, not a cap. Above it the step says the count is untested, and the memory estimate decides, up to the engine's 16: a joint verify pass holds 32 rows and each chat keeps 2. The memory bar and a line with one extra chat's cost follow every change. The step names a count that does not fit, or one the engine does not take, next to the field; Next stays on the step and POST /v1/setup/apply refuses it (over_budget, out_of_range). The extra chats' window names chats 2 to N.
• Both windows, the longest conversation and the one of the extra chats, are steppers too (VIS-199). Minus and plus walk 8,192, 16,384, 32,768 and on, doubling up to the model's own maximum (1,048,576 on Super VL, whose card now says so); the field takes any number, 131,072, 128k or 1m, and shows the short form beside it. A value above the model, below its shortest window or off its step stays as typed with the limit named ("This model holds at most 1,048,576 tokens."), and Next stays on the step. The extra chats hold at most the first chat's window, as the engine cuts a side slot to slot 0's, and the step says so.

Models and engine

• Nemotron 3.5 Super VL takes a window up to 1,048,576 tokens, its checkpoint's own, for slot 0 and the side slots (default 262,144). Checked to 249,983 prompt tokens (five needles of five at 131,054 and 249,983 on EXL3 4.0, slot 0 at 1,048,576, 4 slots); a 1,000,000-token prompt is not measured. The setup guide reads every model's largest window from its checkpoint's config when the weights are on the box, the card's otherwise, and every card now names its checkpoint's maximum: Super VL 1,048,576; Qwen3.8-27B, Flash-Next, Bolt and Lightning 262,144; LFM2.5 131,072. Super VL's memory estimate counts LiLiCorr (3.9 GiB), the measured prefill workspace (3.4 GiB) and a side slot's part beyond its KV (1.1 GiB); slots.max 3 is what was measured (ENG-363).
• NemotronH's prefill workspace is flat (3.34 GiB at 131,054 tokens, 3.18 at 249,983): the memory policy no longer refuses a long prefill by the 27B's line of 30 kB a token, which put 29 GiB on a 1,000,000-token prompt. A slot 0 window above 262,144 adds its KV above it to the policy's ceiling and peak, as the side slots' bytes are added (ENG-363).
• NemotronH keeps attention's partials once per power-of-two bucket of verify widths and slot instead of once per width: at a 1,048,576-token window a set is 32 MiB a row, and the 33 widths could hold 17 GiB. NemotronH and LFM2.5 keep at most 32 joint decode graphs (one per set of slots decoding together and their attention classes), the least recently used dropped and captured again on demand; the count had no bound with more slots and longer windows. /health reports them under parallel.joint_graphs. The bound is a safe pick, not a measurement (ENG-363).

Upgrade: run the installer again; chatd then starts with the engine, and bin/tandem, run/ops/serve.env and the scripts in run/ops are rewritten. A chatd started earlier keeps the old code until tandem chat restart.

0.11.1, 2026-10-16

0.11.1 fixes how chatd, the Chat tab's server, reaches the folders a chat's sandbox can write; update every box that runs the Chat tab. It gives Bolt's EXL3 packs 2.5 to 6.0 their StairCut price files, so drafting no longer runs slower than plain decoding on them, takes two contributions, names each weight pack's maker in the setup guide, and describes the install through the setup guide. Every model writes the same text as in 0.11.0.

Security

• Chat tab: chatd saves an attachment under /workspace/uploads/ without following a symlink. The sandbox can write that folder, so it could make uploads/ a symlink, or leave one at the file's name, and chatd then wrote the user's file wherever the link pointed on the host. Now uploads/ must be a real folder and the file a new one; when chatd cannot save it there, it sends the message all the same and the prompt says why. chatd cuts a name over 255 bytes, keeping its extension; on Linux such a name failed the send (SEC-1). Reported by Joey.
• Chat tab: chatd's other reads and writes in the folders the sandbox writes into do not follow a symlink either: the file tools and files pane, snapshots with their undo, and the Open WebUI runner's files in the sandbox's home. Before, a symlink at the runner's run.py, the toolkit file or the request file made chatd overwrite the link's target, and a snapshot copied a file the sandbox swapped for a symlink into the chat. Each access walks from the folder's root by directory fds opened with O_NOFOLLOW, a write goes to a new file renamed into place, and a read takes a regular file only (a FIFO blocked chatd). chatd replaces a tool call id from the backend that is not a plain name before the id names a file (SEC-1).

Contributions

• Asset fetches download a pinned asset at its revision (SPD-121). ops/fetch_weights.py planned an asset at its revision but fetched the repository's default branch and checked it against that branch's manifest. Contributed by joeynyc (pull request #1).
• NemotronH's tier fingerprint counts its GGUF reader and loader (gguf.py, gguf_load.py) and the GGUF file itself (QSE_NH_GGUF, a .gguf in the model folder): another GGUF restored the first one's saved states (ENG-327). The NemotronH profiles' fingerprint changes, so Bolt's and Super VL's saved conversations from 0.10.0 and 0.11.0 are not reused: their first request reads the prompt again. Contributed by joeynyc (pull request #2).

Nemotron 3.5 Bolt: StairCut price files

• Bolt's EXL3 profiles 2.5, 3.0, 3.5, 4.0, 5.0 and 6.0 ship StairCut price files (ops/prices/nemotron-3.5-bolt-74b-a9b/, SPD-127), measured on the Spark: the verify chain of 1 to 16 rows at 1,024 and 8,192 tokens, the plain step, and a spec prior for a drafted round, (8-row chain + draft) / (0.75 x LiLiCorr's accepted length), 13.9 to 20.2 ms a token. tools/nh_check.py prices --lilicorr writes it.
• Without a file the cut priced a one-row block at 1 ms and took the anchor alone after the draft had run, wherever the draft was unsure; on 0.11.0, 3.0 to 4.0 wrote prose at 31 to 33 tok/s and chat at 39 to 40, below their plain 40.5 to 41.7.
• Setup Guide rows on 0.11.1, one request, 512 greedy tokens, thinking off, 3 runs of the profile's drafting beside 3 plain runs; plain / drafted prose / chat / code, tok/s: 2.0 43.2 / 65.6 / 71.1 / 102.1; 2.5 41.9 / 65.8 / 68.9 / 114.7; 3.0 41.6 / 58.1 / 69.3 / 100.8; 3.5 40.6 / 59.5 / 65.9 / 101.0; 4.0 40.4 / 52.0 / 62.0 / 97.9; 5.0 33.9 / 51.7 / 55.5 / 81.7; 6.0 29.1 / 40.1 / 50.1 / 76.3; the IQ2 GGUF with the lookup 27.7 / 27.7 / 27.8 / 31.9. Every drafted text is the plain run's. DGX Spark, 2026-10-16.
• Super VL keeps its 0.11.0 price files. On 0.11.1 NVIDIA's NVFP4 drafts above plain on every ask (prose 23.7, chat 28.5, code 39.5 against 19.6). The EXL3 2.0 pack's prose varies from run to run: two runs at 37.3 tok/s, one at 33.5 with the cut on plain steps, against 36.5 plain; chat and code drafted 45.4 to 63.7, one code run 44.6 (cause not isolated).

Setup guide and install

• Each weight card names its source: a TandemLLM quant, a vendor release or a community quant, the maker and the Hugging Face repository as owner/name (VIS-175). Every profile in models/*/model.yaml carries made_by. Flash-Next's community section lists turboderp's five packs from models/community-builtin.json, which ships with the release and is never fetched; Use selects the matching flashnext-<bpw> profile.
• The install is two commands, then the setup guide in the browser: bash install.sh --setup --yes and http://127.0.0.1:8000/dashboard/#/setup (OPS-149). README, docs/setup.md, the manual and install.sh's last message say so; use-profile.sh moved to "Advanced: from the command line".

Upgrade: run the 0.11.1 installer again (--setup --force-setup on a box installed with --setup, the same --dir otherwise). chatd is its own process and keeps the old code until it restarts: bash ~/TandemLLM/src/ops/chatd.sh stop, then bash ~/TandemLLM/src/ops/chatd.sh start.

0.11.0, 2026-10-16

0.11.0 adds a fifth model, NVIDIA's Nemotron 3.5 Super VL 120B-A12B, from our EXL3 packs at 4.0 and 2.0 bpw or NVIDIA's NVFP4 release, with NVIDIA's LiLiCorr drafter for Super VL. The other models' text and speed do not change, and their saved conversations carry over.

Nemotron 3.5 Super VL 120B-A12B

Super VL is a NemotronH hybrid (ENG-E13): 40 Mamba2, 40 mixture-of-experts (512 experts, 22 a token, one shared) and 8 attention layers, an MTP head, 1,048,576 tokens of context (TandemLLM serves up to 262,144). Text only: the vision tower is not served yet. Reference: docs/nemotron-h.md.

• ModelOpt checkpoints read as stored (ENG-351): NVFP4 projections (uint8 codes, e4m3 group scales, an fp32 tensor scale) and FP8 projections (e4m3 codes, an fp32 tensor scale), audited against the layer table. The routed experts of a layer run as one NVFP4Bank (tools/nvfp4_relu2.py: one program a row-expert pair, fp32 products, row-invariant), FP8 projections as FP8Linear (a 16-row tensor-core product, the scale on the fp32 sum). Static activation and KV scales are read past: activations and the KV cache stay in BF16.
• LiLiCorr with its own head (ENG-353): Super VL's drafter carries an NVFP4 LM head over the target's vocabulary and reads five taps with a window of 8,192. Accepted length 4.82 on NVIDIA's NVFP4, 4.65 on the EXL3 4.0 pack (4 prompts of 192 tokens, thinking on).
• Profiles nemotron-super-vl-exl3-4.0 (default for Super VL), nemotron-super-vl-exl3-2.0 and nemotron-super-vl-nvfp4, each with LiLiCorr and the lookup cut by StairCut, and the model's Setup Guide card (ENG-355). The price files carry a spec prior, so the cut drafts from the first request.
• EXL3 packs converted from NVIDIA's BF16 checkpoint (revision 97e00975) by exllamav3 1.6.2 with a patch for Super VL's layer layout (ENG-347, ENG-349): 4.0 has every projection at 4 bits, head 6, MTP head 4 (65.0 GB); 2.0 has routed experts at 2 bits and every other projection at 4 (36.84 GB). Each is in a public Hugging Face repository pinned by commit, with its model card, NVIDIA's OpenMDW-1.1 license text, a NOTICE and the converter patch (ENG-359).
• Against the BF16 checkpoint's logits (tools/nh_quality.py, 8 wikitext-2 rows of 1,024 tokens): EXL3 4.0 KL 0.049, top-1 92.9 %; EXL3 2.0 KL 0.325, top-1 82.1 %; NVIDIA's NVFP4 KL 0.041, top-1 93.4 %.
• Through the server, one request, 512 greedy tokens, thinking off, mean of 3 runs: plain 31.1 (4.0), 36.5 (2.0) and 19.6 tok/s (NVFP4); with LiLiCorr and the lookup on prose / chat / code / edit / copy 33.1 / 43.9 / 53.6 / 101.0 / 108.3 (4.0), 35.6 / 46.2 / 46.3 / 125.1 / 139.2 (2.0), 23.9 / 28.7 / 39.9 / 73.5 / 81.7 (NVFP4). Every drafted text is the plain run's. vLLM on NVIDIA's NVFP4: 19.4 plain, 29.9 / 34.8 / 52.7 / 72.4 / 72.7 with LiLiCorr. DGX Spark, 2026-10-16.
• Not yet (ENG-357): prompts read at 476 to 589 tok/s, 4 to 5 times slower than vLLM; with a drafter, requests at once take turns instead of sharing a pass. The vision tower and the MTP head are not served.

Upgrade: run the 0.11.0 installer again (--setup --force-setup on a box installed with --setup, the same --dir otherwise), then ops/use-profile.sh nemotron-super-vl-exl3-4.0 for Super VL.

0.10.0, 2026-10-15

0.10.0 adds a fourth model, NVIDIA's Nemotron 3.5 Bolt 74B-A9B, from our EXL3 packs at 2.0 to 6.0 bpw or NVIDIA's IQ2 GGUF, with NVIDIA's LiLiCorr drafter, and limits each Chat tab sandbox to 1 GiB. The other models' text and speed do not change, and their saved conversations carry over.

• Chat tab: each chat's sandbox has 1 GiB of memory (QSE_SANDBOX_MEM_GB; was 4 GiB), and four of them take what one took (QSE_SANDBOX_TOTAL_GB follows the limit). A process the kernel kills at the limit fails its call with "out of memory (1 GiB sandbox limit)", read from the container's cgroup, also when a pipe's writer exits 1 after its reader was killed; the call card shows it (SRV-485, VIS-167). What fits in 1 GiB, measured: docs/chat.md.
• Chat tab: an attachment is never inlined past the window. Every attachment is saved under the chat's /workspace/uploads/ when its message is sent; one message's text files share a budget of a quarter of the free window, at most 32,000 tokens, and a file past it is one line with its name, size, tokens and path. The chip shows size and tokens on upload and marks a file that goes as a file. A request that does not fit names the attachment or the largest part instead of advising a new chat (SRV-487, VIS-169).
• Engine: POST /v1/cache/release {bytes} (as /v1/cache/clear: the admin token or trusted-local) gives back what the engine holds idle, cheapest first: free allocator blocks and caches while slot 0 is idle, the small models, idle side conversations, slot 0's idle conversation (to the NVMe first). chatd asks it before a sandbox waits and before the memory guard stops one (SRV-483).

NemotronH (Nemotron 3.5 Lightning, Bolt)

A fourth model family: NVIDIA's NemotronH hybrid (ENG-E12), Mamba2, NoPE attention and mixture-of-experts blocks with an MTP head, built on Nemotron 3.5 Lightning 30B-A3B BF16 for Nemotron 3.5 Bolt 74B-A9B, served from NVIDIA's IQ2 GGUF or EXL3 packs at 2.0 to 6.0 bpw, all with NVIDIA's LiLiCorr drafter. The other models' text and speed do not change. Reference: docs/nemotron-h.md.

• engine/models/nemotron_h/: configuration (hybrid_override_pattern, layers_block_type, block_configs with per-block top k, width and expert count, Omni llm_config), reference forward, BF16 and GGUF readers, Triton row kernels, board engine, MTP head, serving adapter.
• Correctness: reference within 2e-5 of transformers' NemotronH (three tiny models, fp32). Lightning BF16 against transformers BF16: top token 98.63 % of 1,024 chat positions, all 14 differences near ties.
• Exactness: a verify row equals the decode step's row bit for bit; a commit walks each SSM state along the kept path with the decode kernel, so the committed state equals the decode steps'; load-time self-check over 24 rows and the states.
• GGUF: nemotron_h_moe files, IQ2_XXS weights and expert banks decoded on the fly (tools/iq2_linear.py, twin equal to gguf-py bit for bit), Q8_0 dequantised at load; other types refused by name.
• Profile nemotron-lightning-bf16 (test model, Setup Guide order 90): decode step 33.63 tok/s (llama.cpp on the same BF16 weights 33.31); edit and copy asks with the lookup 105.8 and 140.7 tok/s; 1, 2 and 3 requests at once 33.7, 44.7 and 50.7 tok/s in total. DGX Spark, 2026-10-15.
• The checkpoint's MTP head as a draft head (QSE_NH_HEAD=mtp, off in the profile): its draft is the trunk's greedy token at depth 1, 2, 3 in 91.9, 80.6, 69.5 % of 1,000 chat positions.
• NVIDIA's LiLiCorr drafter for Bolt (SPD-123), on by default in the EXL3 profiles beside the lookup (the IQ2 profile drafts with the lookup alone: with LiLiCorr beside it StairCut kept plain steps on this build, cause not isolated; its edit and copy asks run at 97.9 and 131.2 tok/s with the lookup, 27.4 plain): the engine keeps the residual rows of six layers in a ring a state (inside the graphs, held by the self-check); the drafter's block, sliding-window attention with sinks, dynamic convolution, candidate lattice and walk run as one CUDA graph on NVIDIA's NVFP4 checkpoint. Accepted length 4.55 on 8 chat prompts (llama.cpp on the same prompts 4.57). Every drafted text is the plain run's; a verify block walks the SSM once a head.
• Nemotron 3.5 Bolt 74B-A9B from NVIDIA's own GGUF (ENG-325, ENG-327, SPD-121, VIS-163): profile nemotron-bolt-iq2 and its Setup Guide card. The GGUF is read as stored: IQ2_XXS experts, NVFP4 Mamba2 projections (ggml type 40, with their per-tensor scales) repacked without a value changed onto the NVFP4 tensor-core kernel, Q8_0 head and attention on a tile kernel. Against llama.cpp on the same file: top token 97.75 % of 1,024 chat positions, all 23 differences near ties. Decode step 27.94 tok/s (llama.cpp 35.37); edit and copy asks with the lookup 57.7 and 58.9 tok/s; 1, 2 and 3 requests at once 26.9, 32.6 and 38.3 tok/s total; 262,144 tokens of context. Bolt has no MTP head. A CUDA GEMV for the IQ2_XXS shared experts is not in this release: with it the decode step measured 32.7 tok/s and LiLiCorr with the lookup prose 40.5, code 73.5, chat 48.0, edit 101.8, copy 147.9 tok/s (512 greedy tokens, thinking off), so this profile decodes and drafts slower than those rows. DGX Spark, 2026-10-15.
• Nemotron 3.5 Bolt 74B-A9B as an EXL3 pack (ENG-335, ENG-337, ENG-339, SPD-125): profile nemotron-bolt-exl3-2.0 and its Setup Guide card, converted from NVIDIA's BF16 checkpoint by exllamav3 1.6.2 with a patch for Bolt's layer table: routed experts at 2 bits, the Mamba2, attention, latent and shared-expert projections at 4, head 6 (23.48 GB, 20.22 GiB at load). Against the BF16 checkpoint's logits (tools/nh_quality.py, 8 wikitext-2 rows of 1,024 tokens): KL 0.170, top-1 84.1 %, perplexity 6.81 (NVIDIA's IQ2 GGUF 0.259, 79.6 %, 7.48). A verify block's EXL3 projections decode each trellis tile once for 16 rows (tools/fn_kernels.py dense_rows, every output byte the one-row dense call's) and the expert bank reads an expert once for a group of its pairs: an 8-row chain 33.8 ms (74.7 a CTA a row). Decode step 44.2 tok/s; with LiLiCorr and the lookup, 512 greedy tokens, thinking off: prose 65.8, code 99.0, chat 69.3, edit 152.8, copy 256.0 tok/s, every text the plain run's; 3 requests at once 72.1 tok/s in total. DGX Spark, 2026-10-15. On this release through the server, 3 runs each (ENG-345): plain 43.3 tok/s; with LiLiCorr and the lookup prose 65.7, code 102.8, chat 69.4, edit 180.5, copy 244.7.
• Nemotron 3.5 Bolt 74B-A9B as EXL3 packs at 2.5, 3.0, 3.5, 4.0, 5.0 and 6.0 bpw (ENG-337): profiles nemotron-bolt-exl3-<bpw> with LiLiCorr and the lookup, no StairCut price file yet. Against the BF16 logits KL 0.094, 0.061, 0.044, 0.034, 0.016, 0.0087 and top-1 87.3 to 96.2 %; decode step 39.9 to 27.8 tok/s (one CTA a row). On this release through the server, 3 runs each (ENG-345): plain 42.1, 41.7, 40.6, 40.5, 34.1 and 29.2 tok/s; on 4.0, StairCut keeps plain steps with LiLiCorr (40.3 to 40.5 tok/s on every ask, cause not isolated). DGX Spark, 2026-10-15 and 16. Every Bolt EXL3 pack is in a public Hugging Face repository pinned by commit, with its model card, NVIDIA's OpenMDW-1.1 license text and the converter patch, in one collection.
• Not yet: Bolt's RADIO image tower (its GGUF mmproj, the pack's BF16 tower); TandemLLM serves Bolt as text only.
• Asset fetches may pin a revision, and a profile that names its files in QSE_ASSETS fetches a missing one from its registry entry (ops/use-profile.sh).
• Saved conversations (ENG-341): fingerprint v2 leaves out modules under engine/ that only another model imports (the LiLiCorr draft head), so the 27B's v2 and v3 code fingerprints stay 0.9.x's. tools/fn_kernels.py gained dense_rows (SAME_ROWS maps its bytes to 0.9.x's), and the legacy fingerprints now read every mix of the bytes SAME_ROWS replaced, so the first Flash-Next or LFM2.5 start adopts the files 0.9.x wrote, once.

Fixed

• Chat tab (SRV-481): a tool call waited for its sandbox without end. On the Spark every bash call of a chat waited 21 minutes on a 30 GiB start line (QSE_SANDBOX_MIN_START_GB, a leftover from the Flash-Next memory fits) and a 19 GiB reserve, with MemAvailable at 20 GiB. The start line is gone. A sandbox starts when MemAvailable less its limit stays at 4 GiB or more (QSE_SANDBOX_FLOOR_GB): the engine keeps itself at its own floors, and a prefill's workspace measured 2.7 GiB at 203,052 tokens. The guard stops an idle sandbox under 3 GiB and kills them all under 2 GiB. A wait ends after 60 s (QSE_SANDBOX_WAIT_S) with the reason and the numbers, as the tool result the model reads and on the call card; the terminal and stdio MCP servers no longer wait forever either.

Upgrade: run the 0.10.0 installer again (--setup --force-setup on a box installed with --setup, the same --dir otherwise), then ops/use-profile.sh nemotron-bolt-exl3-2.0 for Bolt.

0.9.2, 2026-10-15

0.9.2 fixes two install bugs an early tester hit on a new DGX Spark with 0.9.1. The engine, the conductor and the dashboard do not change.

Fixed

• ops/fetch_weights.py (OPS-143): verify() compared a fetch entry's globs to file names literally. A download whose list is all globs (Qwen3.5-0.8B for the shadow reader, Qwen3.5-2B for the narrator) checked no file and failed with "0 files checked: FAILED"; Laya's files under multilingual/ and typed-decisions/ were not checked. The globs now select the files, as in the plan.
• install.sh --setup (OPS-143): the installer wrote QWEN38_MODEL, the Hugging Face cache's base folder, after the weight profile. On a new box that folder is empty, the 27B's EXL3 profiles did not switch to base-plain, and use-profile.sh exl3-* refused with base-plain on disk. The keys the installer adds now come before the profile, and an EXL3 profile takes base-plain when that folder holds no snapshot. A standalone community pack reads its own folder on an install.sh box for the same reason.
• install.sh with HF_HOME or HF_HUB_CACHE set (OPS-143): serve.env names that cache, and use-profile.sh passes it to the fetch, so a later shell without the variable fetches and finds the base in the same cache.

Upgrade: run the 0.9.2 installer again (--setup --force-setup on a box installed with --setup, the same --dir otherwise). It rewrites run/ops/serve.env and the scripts in run/ops; a git checkout of the tag alone keeps the old ones.

0.9.1, 2026-10-14

0.9.1 adds web search and media playback to the Chat tab, turns on the loop guard, the loop escape and memory notes in the shipped conductor profiles, and fixes the Setup Guide's install page and the EXL3 base download. The decode path does not change: under engine/ only the small models' slot changed (VIS-153).

• Chat tab: a built-in web_search tool with DuckDuckGo and SearXNG providers, set in the chat settings (SRV-469, VIS-157). Reference: docs/chat.md.
• Chat tab: video, audio, PDF and other binary files open in a player, the browser's PDF viewer or a size line with Download, in the Files pane and the present card; chatd's view route streams a file with byte ranges (206, 416, ETag) and no 2 MiB cap (VIS-159).
• Conductor: the same-call guard (C3) is on in the shipped coding, chat-app, hermes, overnight and chat profiles (toolgate.same_call: true; chat's C3 acts). C3 counts only the current user turn, a run whose result took the client 10 s or more is a wait (wait_s), and calls are held only on a request whose turn already ends with the repeat; every other turn streams its calls as before. A one-time migration turns it on in each stored profile whose versions never had it on (SRV-473).
• Conductor: the loop escape. After a repeated or near-repeated tool call the session decodes with the model card's sampling (the checkpoint's generation_config.json, fresh seed) and C3's escape note until it makes progress, then greedy again; the client's own sampling wins. Profile section sampling (loop_escape, agentic_default: greedy | card; shipped greedy, escape on in the agent profiles). Ledger sampling loop_escape / card, usage tandem.sampling, /health sampling.card and sampling.loop_escape (SRV-475). Output changes on those turns only.
• --reasoning-effort none and off: thinking off for every request that says nothing about thinking (SRV-477).
• Conductor: the shipped chat, coding, hermes and chat-app profiles have memory: on, and a one-time migration adds it to each stored one without the switch (an explicit value stays; general keeps none); the memory card says the notes are the model's memory between chats, and a profile with memory: off tells the model so; the Context panel shows a turn's memory notes (SRV-471). Memory-on prompts change by the card's new sentence.

Fixed

• Setup Guide, install page (VIS-149): a reload no longer shows a finished job as running. The job stream sends the log on disk, the job as it is now, then its events; the page keeps a final job final and handles no event after it closes the stream.
• Setup Guide, install page (VIS-151): a finished job keeps its whole log after a reload (the stream ends at the job's done event); past 300 lines the log box says how many it dropped from the top; a job whose folder is gone says so instead of an empty log.
• Setup Guide, review and install page (VIS-155): the change list compares every key the current run/setup.env sets too, so a model switch lists the keys it drops (LFM2.5's TEMPERATURE, TOP_K and REP_PENALTY: 16 changes, not 13) as removed, with what the engine runs with then; the install page lists the changes under the check step.
• Setup Guide, fetch and smoke steps (VIS-153): the fetch step downloads the models of the tools the selection turns on (Qwen3.5-0.8B for the shadow reader, Qwen3.5-2B for the narrator, Laya), and the review lists every download with its size; a tool model nothing downloads is a missing_assets error. After the start the smoke step reads the tools' states: a failed one is a warning on the job with its reason. A failed shadow reader load is tried again after 300 s in an idle window; /health roles carry the load error.
• EXL3 profiles of Qwen3.8-27B (OPS-133): their base is base-plain, the Qwen3.8-27B-FP8 tensors an EXL3 load reads (embedding, norms, GDN gates and convolutions, vision tower, config, tokenizer), fetched by HTTP range requests into ~/.qwen38-spark-engine/weights/base-plain: 3.54 GB instead of 30.41 (the F8_E4M3 projections, their scales and lm_head.weight stay out). A fresh exl3-6.0 downloads 36.48 GB, not 63.34. A box with the full FP8 snapshot uses it and downloads nothing; NVFP4 still needs the full base.
• Dashboard service worker (OPS-131): a page load that the reverse proxy answers 502, 503 or 504 gets the cached shell, as a network error did; the page says the engine is restarting and reloads once /health answers.

0.9.0, 2026-10-14

A third model: LiquidAI's LFM2.5-24B-A2B (ENG-E10), 24B parameters, 4 of 64 experts a token, 30 short-convolution and 10 attention layers. The Qwen models' text and speed do not change. The shared EXL3 kernels' bytes changed (half-integer rates); SAME_ROWS maps them to the old bytes, so the first Qwen start adopts its saved conversations once. Reference: docs/lfm2.5-24b-a2b.md.

LFM2.5-24B-A2B

• engine/models/lfm2_moe/: configuration, reference forward, pack reader, Triton row kernels, board engine. Decode step, verify block (up to 32 rows) and commit run as CUDA graphs; a verify row has the decode step's bits, checked by a self-check at load.
• Correctness: reference forward within 2.1e-5 of transformers' Lfm2MoeForCausalLM (tiny model, fp32). BF16 engine against transformers: top token 98.05 % of 1,024 chat positions, 19 of 20 differences near ties, KL 0.0064. 4.0 pack against exllamav3: 97.85 %, all 22 differences near ties. Trellis decode equals exllamav3 LinearEXL3 bit for bit.
• Profiles: lfm25-bf16 (Liquid's checkpoint) and lfm25-exl3-{2.5,3.0,3.5,4.0,4.5,6.0} (exllamav3 1.6.0 packs, public Hugging Face repositories pinned by commit, no token needed). Each: 131,072-token context, three slots.

DGX Spark, 2026-10-14. Decode step: one request, 256 tokens, from its graph. Chat: one greedy request through the server, 512 tokens, thinking off, plain / lookup. Quality: exllamav3 model_diff against BF16 on oasst1 chat. Not the benchmark row. Raw runs: models/lfm2.5-24b-a2b/measurements/0.8.3-summary.json.

• Default lfm25-exl3-3.5. Rule: the fastest profile in chat as served (lookup on) with chat top-1 against BF16 of at least 85 %. 3.0 is faster (168.5 tok/s) at 83.6 %.
• No -hq profile: an exllamav3 -hq 4.0 served chat 9 % slower than 4.0 at a worse KL than 4.5 (0.082 against 0.063).
• Dense splits per pack (ops/lfm25-dense-splits.json). Decode step without / with, tok/s: 2.5 171.2 / 188.2, 3.0 159.1 / 175.0, 3.5 143.4 / 153.5, 4.0 136.1 / 145.9, 4.5 121.6 / 128.8, 6.0 106.9 / 106.9.
• Server: the checkpoint's chat template. Thinking off, from a client or the conductor, closes the <think> block the template always opens. Pythonic tool calls (<|tool_call_start|>[f(a=1)]<|tool_call_end|>) become OpenAI tool_calls; values are parsed as Python literals, never run. Default sampling when a request names none: temperature 0.1, top_k 50, repetition penalty 1.05.
• Conductor, Laya and the narrator on by default, with the model's fit to the router (SRV-441, docs): Laya alone answers the class question, a turn nobody classified stays lean, and a class sets the level and budget with only its core skill in the note. On 2.5, raw / router as written / fit: four short chat asks 2,155 / 9,098 / 2,234 reasoning tokens; seven Chat-tab tasks run twice 14/14 / 12/14 / 14/14, 0 / 9 / 0 tool errors in the last two.
• The Chat tab's present tool: asked to open a page it wrote, the model calls it again. The router's testing packages had it check pages through the shell (6 follow-ups: 145 tool calls, 15 shell calls in the follow-ups, 113,211 reasoning tokens); with the fit 20 calls, present in all 6 follow-ups, 7,228 tokens.
• Three slots: 1, 2 and 3 requests at once decode 139.9, 169.1 and 177.8 tok/s in total on 4.0; each text equals its request alone. When every slot takes a plain step, one joint graph reads the weights once.
• Disk tier and session spill: 12,288 tokens restored from the disk in 0.96 s against 2.6 s cold, with the cold answer.
• Not on LFM2.5 yet: a draft head (none published), the shadow reader (it reads Qwen's token ids), the KV pool (off in the profiles, not verified on this model).

The lookup without a draft head

LFM2.5 drafts with the lookup alone (NoHead in the shared tree drafter), cut by StairCut.

• A round with no lookup match is the plain step and is priced as one. Each round's own match decides; StairCut's class-wide skip is not consulted.
• The lookup's confidence carries its match length, so a long copy match scores above a short phrase match. BF16, store off: edit and copy 153.7 and 157.8 tok/s against 44.3 plain.
• One StairCut price file per profile (ops/prices/lfm2.5-24b-a2b/). Without one, the lookup cost the 2.5 pack 26 % on prose.

Effect: prose, chat and code within 5 % of plain; edit and copy asks 1.8 to 3.6 times faster; every text equal to plain decoding.

Fixed

• A pack stored as one model.safetensors without an index counts as complete in ops/use-profile.sh and the Setup Guide. Before, a switch to the 2.5 pack downloaded it again and refused it.
• Reading a slot's kv.length commits nothing. Before, the health report's read committed a joint plain step twice with three slots ("LFM2 forward at 53, the state holds 54 rows").
• Attention split buffers are kept per attention class. Before, a verify graph captured at a new class freed buffers an older graph replays into (illegal memory access past 2,048 keys).
• A suffix store written by another tokenizer turns the store off instead of stopping the start.
• The dashboard's disk tier default is 16 GB, as the server's since 0.4.3 (it showed 50).
• /v1/completions with a text prompt and two or more slots answers. Before, the slot choice read the prompt as a tensor and the request failed with 500 (SRV-443).
• Chat tab: the plus menu draws outside the tab strip, which clipped it (VIS-133). A page a write_file or edit_file call left opens from its tool card, the plus menu's Preview and the Files viewer (VIS-135).
• Chat tab, Context panel: it counts every tool call on the branch, the files the file tools carried, the routed packages' tokens, the chat's KV rows, the resident prefix and the sandbox's memory (SRV-445, SRV-447, SRV-449, SRV-451, SRV-453). A figure it cannot show names the reason instead of a dash (VIS-137). The packages and the prompt split survive a chatd restart (SRV-455).

Shared EXL3 kernels

• Half-integer rates (1.5, 2.5 and 3.5 bits a weight, tiles 16 K + 8 int16 wide), as exllamav3 1.6.0 writes them below 4 bpw, in the decode GEMV, the MoE bank, the prefill grouped GEMM and the slab decode. 7-bit experts in the MoE kernels.
• Whole rates unchanged: 51 of 51 kernel outputs equal the previous tree's bit for bit.

Setup Guide

• Wren runs beside LFM2.5: on 3.5 it loads its own Qwen3.5-2B in 20 s (3.62 GiB), as the model has no narrator copy to share.
• The Tools step no longer offers turning vision off on a text-only model.

0.8.3, 2026-10-13

Repository hygiene for the public release: no private hostnames, home paths or personal notes in the files; no code change. The engine's modules are the bytes of 0.8.2, and the text does not change.

• Comments, docs, tests and this file's history no longer name a person who gave an instruction, and no agent as the author of code or docs.
• Tests and e2e specs use 192.168.1.x for a LAN address and tandem.example.lan for a host behind a proxy.
• install.sh installs v0.8.3 by default; the install commands in README.md, docs/setup.md and the manual follow.

0.8.2, 2026-10-13

0.8.2 fixes two bugs found on v0.8.0 in Open WebUI on a phone: a chat that waited minutes for slot 0 while the side slots were free (SRV-409), and Flash-Next calling itself Fable or Claude, made by Anthropic (SRV-411). The engine's modules are the same bytes as in 0.8.1; the disk tier's fingerprint does not change (the new knob is the server's).

• A request whose conversation slot 0 holds waits for slot 0 at most 3 s (QWEN38_SIDE_SLOT0_WAIT_S), then takes a free side slot it fits in. Live on v0.8.0 a new Open WebUI chat waited 437 s behind another chat's 496 s write_file generation while both side slots were free. It had matched another chat's NVMe entry at 8,192 tokens, which is only Open WebUI's system prompt and tool list, and the wait for slot 0 had no limit. The spill wait now needs an entry that reaches past the request's first user message, as the view and resident rules have since SRV-223; the log says spill_head= for a match at the client's head. On the side slot the prefill reads what the disk tier holds and the rest again. A request too long for every side slot still waits for slot 0.
• /health parallel.max_running is now the most requests that may run at once: every slot built, fewer only while the admission holds requests for memory, with max_running_reason. It used to be the high-water mark of busy slots (max_running: 2 on three slots read as a cap); that number is peak_running now. New: requested, spill_waits, slot0_gave_up; slot0_grants counts again.
• The Lanes view's queued line says why a request waits: "Queued: all 3 slots are busy", "Queued: slot 0 holds its conversation; a free side slot takes it after 3 s", "Queued: too long for a side slot (65,536 tokens); waits for slot 0". While the router's questions run it says "Queued: TandemLLM decides how to handle the request first". The live feed's queue object carries the same reason.
• Asked who it is, the model now says it is TandemLLM running the served model on the user's own machine. On v0.8.0 Flash-Next answered "I'm Fable, an AI made by Anthropic" and "I'm Claude": the reference box served it with the conductor off (box values carried over from the first Flash-Next test launcher), so the identity note never went in, and Open WebUI's system prompt names only tools and skills, one of them named after another assistant. The note names the model from the served pack's model card (Qwen3.8-Flash-Next or Qwen3.8-27B; before, the note called Flash-Next Qwen3.8-27B), says it is no other assistant and not made by Anthropic or OpenAI, and goes in for every client the conductor serves: Open WebUI, opencode, the Chat tab, Hermes and any other. It stays at the end of the asking message, where the prefix cache keeps it.
• The reference box runs Flash-Next with the conductor on, with the same store and profiles as the 27B.
• Standalone EXL3 checkpoints of the 27B family load directly (#1, 0xWhiteMage): a complete BF16-origin mul1 EXL3 pack is passed as --model, with no separate BF16 or FP8 base, its FP16 plain tensors re-encoded as BF16 only where every value survives the round trip (docs/quantisation.md). Vision and quantised MTP are not qualified on that path; it is text only, and speculative parallel serving still needs the lenrouter drafters. The 27B's own profiles (an EXL3 overlay over the base, or NVFP4) take the path they took before, so their saved conversations stay: engine/loader.py's new bytes count as the old ones in the disk tier's fingerprint (SAME_ROWS in engine/methods/manifest.py), the first 27B start adopts the files of v0.8.0's fingerprint once, and the 27B's code fingerprint stays 0.7.2's (9ac3d8a94c8a9199). Flash-Next's fingerprint does not read the loader.
• Live on the reference box (Flash-Next 3.05, three slots): while one Open WebUI chat generated 2,600 tokens on slot 0 for 80 s, a new chat started on slot 1 and the next turn of the chat slot 0 had held moved to slot 2 after its 3.0 s wait; both answered while the long run went on. Asked "Who are you?" and "WHO are you?" with Open WebUI's own system prompt and 49 tools, 5 times each greedy and 5 times each sampled (temperature 0.6, top_p 0.95, top_k 20), 20 of 20 answers said "I'm TandemLLM, running the Qwen3.8-Flash-Next model on your own machine"; v0.8.0 gave 0 of 20 (Fable, Claude, Kiro, Kimi). opencode, the Chat tab and an OpenAI SDK client gave the same answer. Single-request decode, greedy, thinking off, the second of two runs: 54.5 tok/s on v0.8.0 without the conductor, 59.5 tok/s on 0.8.2 with it (the conductor's notes change the prompt, so the texts differ: no slowdown, not an A/B). Before its prefill, a new Open WebUI chat waits for the router's two questions on the served model: the class (440 tokens, about 0.55 s) and the package (2,355 tokens, about 2.0 s).
• The Flash-Next profiles load Laya (laya-english at the start, the other two on demand), as the 27B's box does, because the memory features ask only a decision model: the remember/forget/recall intent on a new user message (about 20 ms) and the background writer's gate. Without it they answered nothing on Flash-Next, so memory did not work there. Live on 3.05 with Laya loaded: "remember that my favourite colour is green" got the intent yes in 20.6 ms and the memory skill attached; the writer's gate answered yes in 20.9 ms. On 3.05 with three slots and the 262,144-token window the residency planner leaves 36.8 GiB after Laya's claim and the engine's growth to its peak (floor 15 plus margin 4). It does not shorten the router's questions: the router document's decider stage has been off on both models since 2026-10-02 (Qwen 0.949 against Laya 0.782 on 216 labelled requests), so the class question goes to the served model. Decode speed with Laya loaded: 46.8 tok/s on a fresh prompt, against 47.6 and 47.9 on the first runs before (different texts).
• The Chat tab has permission modes per chat, with a default in the chat settings (SRV-387, SRV-391). Auto is the new default: a call that stays inside the chat's sandbox runs without asking (bash, files in /workspace, render_page), and a call that leaves it still asks (web_fetch to a domain the chat has not fetched, an MCP tool not marked read-only, an Open WebUI tool that imports a network or process module, and rm aimed at /, /workspace or a home directory). Ask keeps the old prompts. YOLO asks nothing. Deny rules and plan mode's read-only list hold in every mode, and the sandbox is as locked down as before. Each call a mode allowed is logged as tool.auto_approved in the chat's event log, and the run shows its mode. In a test run on a phone, six bash prompts took 20 s of tapping; replayed through the new rules in Auto, all six run without asking. The chat store moves to schema 5 (runs.permission_mode). docs/chat.md, "Permission modes".
• "Always allow" on a chained bash command saves a rule for every part that no rule allows yet, and the prompt names them ("Always allow cd, node, echo, cat"). Before, it offered only the first part's word, which never covered the command, so the prompt came back: one chat saved "cd" 11 times for cd /workspace && node ... > run.log 2>&1; echo "exit:$?"; cat run.log. A second word counts as a subcommand only for programs that have them (git push, npm install), so "grep tilt" saves "grep". The chat store moves to schema 6, which keeps one copy of each duplicate rule. A command with $(...) or backticks still matches no rule and offers no "Always". This matters in Ask mode; Auto runs sandbox bash without asking.
• Community weight packs: one JSON file per pack under models/<model>/community/, checked by the registry (a pinned commit, EXL3 only, text only on the standalone path) and listed in the setup guide after our profiles as community/<id>. The guide fetches the index from the public repository when its weights step opens or on Refresh, never on a timer, and falls back to its last copy, then to the one in the release. ops/use-profile.sh community/<id> downloads a pack at its commit, checks the hub's sha256 and runs it with our drafters. The first entry is ThinkingCap Qwen3.8-27B Kearuga by 0xWhiteMage (EXL3 4.0 bpw, standalone, measured on 0.8.2: 39.6 tok/s prose, 95.9 code, 51.3 chat). docs/community-weights.md says how to add one.

0.8.1, 2026-10-13

0.8.1 lets a new DGX Spark run every 27B EXL3 profile and Flash-Next with public files only and no Hugging Face token (OPS-95). The engine's modules are the same bytes as in 0.8.0, so the text, the speed and the disk tier's fingerprint stay as they were. The changes are in ops/, server/setup/, the registry, install.sh and the docs.

• ops/use-profile.sh fetches the files a 27B profile reads besides its weights before it switches, and with --fetch-only: the K3 drafters and the K4 draft head (0xBakeer/TandemLLM-Qwen3.8-27B-EXL3-drafters), the BF16 drafters (...-DFlash2-b8 and -b16, into train/ft-b8-v2 and train/ft-b16) and the base checkpoint Qwen/Qwen3.8-27B-FP8 (into the Hugging Face cache, without mtp.safetensors). Before 0.8.1 these four had no download, so an exl3-<size> profile could not start on a new box. Each asset in models/qwen3.8-27b/model.yaml now says where its files come from and where they go.
• The fetcher pins every download to the hub's commit, checks each weight file against the hub's sha256, and moves a folder into place only when it is complete. The disk check runs first and counts the assets, the weights after them and the 60 GB margin. A run that stops continues where it stopped. --dry-run prints every file's URL and size, for the weights and for the assets.
• The EXL3 profiles read the BF16 drafters from the repository's train/ folder. An install.sh box pointed CKPT8 and CKPT16 into the Hugging Face cache, whose folder names are commit hashes, so the drafter arms did not find their K3 files. With --setup they were empty.
• A switch from the command line on a box in setup mode removes run/setup-pending first, so the engine starts with the profile. A failed start puts it back and starts setup mode again.
• The setup guide's plan downloads the missing assets in its fetch step and counts them in download_gb. It refuses only a file with no download (missing_assets). Every published profile is public, so no profile asks for a token.
• install.sh installs v0.8.1 by default (it still named v0.4.8, which has no setup mode). With --setup it points QWEN38_MODEL at the base checkpoint's folder in its own Hugging Face cache. Its messages no longer call the repositories private.
• The switch's smoke turn goes to the engine on 127.0.0.1 unless serve.env or the environment sets SMOKE_URL. Before, it went to the reference box's own address. The smoke's model name follows the conductor, as the setup guide's does.
• New box path for testers: docs/setup.md, "A new box from the command line".

0.8.0, 2026-10-13

0.8.0 adds a second model, Qwen3.8-Flash-Next, the Chat tab with chatd, its own chat service, and the cache peek and prewarm the Chat tab's Context panel reads. Flash-Next 3.05 is served with a 262,144-token window and three slots. The 27B's text does not change: its v2 disk-tier fingerprint is the same as 0.7.2's (9ac3d8a94c8a9199).

The Chat tab

• A Chat tab in the dashboard (#/chat) and chatd, the service behind it (python -m chatd on 127.0.0.1:8016, reached through the engine's /v1/chat-app/*). A run belongs to the server: it goes on when the browser closes, and another device attaches mid-run and catches up from the last event it saw (docs/chat.md).
• Tools in a sandbox per chat (a locked-down container, no LAN, a memory guard that gives way to the engine), permission prompts on every device, the terminal, preview, files and diff panes, MCP servers, Open WebUI tool imports, pinned and blocked skills, plan mode with a todo list and Rewind, the PWA and local notifications.
• The Chat tab is in beta: a Beta pill beside it in the rail, in the phone's bottom bar and in the chat's header. Its known limitations are listed below.
• On a phone the bottom bar has four slots, with the other views behind More, and the composer shows every control.
• The model card in /v1/models (SRV-347): what a model can do, read from models/<id>/card.json, and what the process serves (its window, its pack, the vision tower). POST /v1/tokenize counts a chat body's tokens by part.

The cache peek and prewarm (SRV-349)

• POST /v1/cache/peek says where a chat's rows are before it is sent (resident, the RAM park, a snapshot, the NVMe spill, the disk tier, none), how many tokens are warm and the re-read's estimate; POST /v1/cache/prewarm reads a chat back from the NVMe while the engine is idle. Live on the RC the peek named the tier and line of the next request in every case tried (resident, none, nvme_spill), and a prewarmed chat's next turn resumed at its end anchor.
• The usage frame carries tandem: the prompt by part (system, tools, history, the router's notes, images), the router's choices for the turn (harness), and the cache tier the prompt came from.

Found by the RC's live checks

• A restart could leave the engine unusable on Flash-Next: the spill's restore at startup told the MTP drafter its cache was valid before reading any row back, the drafter read pages the KV pool had released, and the GPU faulted (illegal memory access). Every request after it failed. The rows are now committed before a drafter resumes, and it resumes again over the restored rows (the spill, the disk tier and a side slot's carry).
• Every short request the side lane took on Flash-Next failed ('FlashNextCacheState' object has no attribute 'primed'): the lane lends the engine with the 27B's pending commit in mind. The lane is off for an engine whose capabilities say it cannot be lent; such a request waits or takes an idle side slot.
• Flash-Next had no model card, so the Chat tab saw no thinking and no images on it, and sent an attached image as a file. It has its card now.
• A chat that last ran on a side slot came back cold after a restart. The drain now writes each side slot's conversation to the NVMe too, and its next turn reads it back through slot 0.
• The System view said PREFIX_CHUNK 1,024 needed a restart on Flash-Next, whose caches keep their own grid. It shows the grid the engine runs (4,096), locked.
• A request that arrived during a prewarm waited for the whole save and restore (1,246 ms). The prewarm now stops between blocks when a request waits and answers yielded; the request still waits 0.5 to 1 s.
• Right after a turn, a peek found the engine lock held by the turn's end and the idle graph captures, and answered 409. The peek and the prewarm now wait up to 3 s for the lock unless a request waits for it. A parked chat's next turn reports the park, as its peek does.
• A chat waiting for a permission prompt held one of chatd's two run slots, and two other chats ran one after the other. A run gives its slot back while it waits for the answer.
• An iPhone test (2026-10-13): a 21-step run building a clock stopped with "This conversation is about 319,136 tokens" while the engine held 16,980. chatd counted five 300 KB screenshots by their base64 text, sent each screenshot as a new user message (the model then thought the user kept repeating the request), and could not compact a run with a single user message. chatd now counts an image by its pixels, puts a tool's image inside the tool's result, keeps only the newest screenshot, and compacts a long run's earlier steps between two steps. render_page says each screenshot is a fresh render, and the sandbox runs Chromium for any command. The same prompt now finishes in 6 to 9 steps with at most 8,144 tokens of context.
• The disk tier now knows its files by content (the fp4 fingerprint). Before, the 27B moved to a new folder threw away all 29 of its saved files (2.97 GB) because the fingerprint held folder paths. 0.7.2's files carry over in place, or from the old folder named in QSE_DISK_ADOPT_FROM. On the box, Flash-Next kept its 34 saved chats through the switch.
• Under the 27B's conductor the cache peek counted fewer tokens than the turn would send: it left out the memory notes a turn replays. It now replays them, and the peek's prompt matches the sent one from the second turn on.

Chat tab (beta): known limitations

The Chat tab is in beta in this release, and a Beta pill beside it says so. We know of these limits and have not fixed them yet:

• The agent loop does not yet work the way Claude Code does (epic VIS-E10). Its prompts are not kept prefix-stable from step to step (SRV-375). Old tool results and old thinking stay in the context until a compaction (SRV-365). A compaction writes a general summary, not the task's state (SRV-373). Every sandbox command asks for permission; there is no mode that runs them without asking (SRV-387). Nothing notices when the model repeats the same failing approach with different calls (SRV-367). render_page passes on Chromium's console noise (SRV-369). Long tool output has no "see more" paging (SRV-371). Searches run in the main context, with no sub-agents (SRV-381).
• The engine reads a short Flash-Next chat again on its next turn when the answer ended inside a verified block (ENG-235).
• A request that arrives during a prewarm still waits 0.5 to 1 s (SRV-349).
• When memory reaches its ceiling, the engine drops the parked chat without writing it to the NVMe, so its next turn starts cold.
• Images only: there is no video input (VIS-97).
• The 5.05 Flash-Next pack does not load yet.
• Parallel decoding does not raise throughput over one request at a time: three requests together took 27.18 s against 27.03 s one after another, though the later ones start sooner.
• The phone layout's keyboard behaviour is not verified in Safari on an iPhone.

Qwen3.8-Flash-Next

A second model, Qwen3.8-Flash-Next, served from turboderp's EXL3 packs through the same server, caches, ops scripts and setup guide as the 27B. The 27B's text does not change: its modules, flags and disk-tier fingerprint are as in 0.7.2, and every switch added here is off for it. On Flash-Next, greedy and seeded answers with speculation equal plain decoding token for token.

• Five weight profiles, flashnext-2.05 to flashnext-6.05, and a second tile in the setup guide (models/qwen3.8-flash-next/model.yaml). ops/use-profile.sh switches between the 27B and Flash-Next one engine at a time, and rolls back across models. The fetcher takes a branch of the repo, checks the hub's sha256, and links the n-gram table that the 4.05, 5.05 and 6.05 packs share.
• Each pack has its largest window in the registry: 262,144 tokens for 2.05 and 3.05, 208,896 for 4.05 with the vision tower on (212,992 without). The guide's window buttons and the estimate follow it, and a start with a longer --max-len is refused. 3.05 reads its n-gram table cold. Warm, the whole table resident took MemAvailable less the table to 8.4 GiB in a 146,400-token request at a 147,456-token window, under the 15 GiB floor. 6.05 fits no window on one DGX Spark.
• 5.05 does not load yet: its experts use 5 and 7 bits a weight, and the MoE kernels take 2 to 6 and 8. Its card in the guide says so.
• Speculation drafts with the pack's own MTP layer, cut by the shared StairCut. The lookup (the request's text, the corpus, the suffix store) joins the same round as one tree. Sampled requests walk the tree with the 27B's sampler.
• StairCut keeps its prices, the plain step's cost and a drafted round's cost per context class, by wall time. A class whose drafts cost more a token than plain steps takes the plain step, and a turn drafts only when the saving pays for the draft head's backlog. While idle, the server captures the graphs a context class lacks, so the first drafted request at a new length no longer pays 3.4 to 5.1 s of captures.
• All cache tiers hold Flash-Next's state (grid 4,096). A reset keeps the KV rows in place: a short request run between two turns of a long conversation used to zero rows the next turn resumed over. The resident guest stash is on again.
• The KV pool, side slots and a joint verify pass work on Flash-Next, each answer equal to its single-stream answer, and its profiles turn them on (QSE_PARALLEL=3, QSE_KV_POOL=on). Three requests arriving together took 27.18 s against 27.03 s one after the other on fresh servers, with first tokens at 2.2 to 6.1 s instead of up to about 20 s. One request alone decodes at 45.78 tok/s against 45.79 with one slot. The limits are in docs/parallel-requests.md.
• Flash-Next reads images: the pack's vision tower on the EXL3 kernels, image rows with three-axis rotary positions, on in its profiles (VIS-97). Image chats answer the same with the MTP drafter and without, and the text after an image keeps the text's own agreement with transformers (KL 0.019 after 140 image rows, 0.0025 after 2,184).
• The mixers are int8 with one fp32 scale for each group of 128 weights (QWEN38_FN_MIX=int8; int8rows is the old one scale a row). Against the pack's fp16 mixers at 8,192 tokens: KL 0.0031 on code and 0.0188 on prose, inside the kernels' own noise, where one scale a row gave 0.267 on code.
• On Flash-Next the shadow reader stays off, because its outputs were validated on the 27B's reasoning only. The conductor, the narrator and the residency planner run as on the 27B.
• /health and the dashboard no longer issue CUDA work when they count the engine's state. On Flash-Next, a /health poll during a graph capture invalidated the capture and failed the request (seen at the first decode after a 260,800-token prefill).
• Measured on 3.05 (DGX Spark, one request each): MTP alone, one greedy request of up to 512 tokens a class: prose 49, code 85, chat 62, edit 106, copy 102 tok/s. With the lookup: edit 132 and copy 133. The prefill made 571 tok/s at 260,800 tokens, with a workspace of 10.5 GiB there.
• Measured on 3.05 by context, one greedy request a length and its repeat on one fresh server, the second round, against plain decoding: 1k 56.6 and 57.3 against 41.0 and 45.4 tok/s; 32k 34.9 and 55.6 against 33.0 and 42.5; 140k 42.1 and 52.4 against 33.9 and 41.4; at 260,800 tokens the class takes plain turns, 41.0 and 41.0 against 39.6 and 40.5. All 16 texts equal plain decoding.
• Measured at 260,800 tokens, one request a pack: lowest MemAvailable (less a warm table) 25.5 GiB on 2.05 with its table warm, 31.9 on 3.05 and 17.8 on 4.05 with their tables cold; prefill 585, 571 and 581 tok/s.

0.7.2, 2026-10-11

Wren, the setup guide's assistant, answers a question in words again (VIS-91). The text the engine writes does not change: only server/setup/, dashboard/ and the docs changed, and the engine's modules are the same bytes as in 0.7.0, so the disk caches keep their fingerprint.

• On a phone with 0.7.1, Wren answered on the Model step and then called explain("overview"), the name of the step. The server took a step's name as valid, the page refused it, and the page asked again, four times. "Which one is for coding tasks?" got eight lines of "Wren tried something that does not exist", the 4-round stop and no answer. The sheet dropped to a peek and rose again once per call.
• explain now takes only what the page can highlight: a tool, a method or advanced field, a weight profile or a parallelism field. The server and tools.ts check the same set.
• The prompt has the current step: the ids a tool call may name there, and on the Models, Model and Weights steps the measured table with the fastest profile for each task. It also says which profile the box runs. The question about coding tasks found the conductor's "coding" profile before; it now gets exl3-2.5 at 124.17 tokens a second.
• An invalid change gets one repair. If the repair fails, Wren answers in words with no tools in the prompt, and the page shows one notice that nothing changed. The server drops a highlight the page cannot make, with no repair. The page's last round sends "tools": false, so every turn ends in words.
• A turn shows one notice at most, and none when the repair worked or only a highlight failed.
• On a phone the sheet stays open while Wren works. A change shows inside the sheet as a toast with Show me, which closes the sheet and scrolls to the field. A refused call leaves the sheet and the figure alone. The desktop panel is as before.
• On a phone the mini box's landing (0.7.1) started 60 px right of the field and widened the page by 27 px, and the fixed sheet with it, so its close button went off the screen. The setup page now clips that overflow.
• On the narrator's 4,096 rows, a long chat could cut the page's second round down to the tool's result alone, and the chat template threw "No user query found". The trim now keeps the whole turn Wren answers.
• Measured on the box (tests/gpu/setup_wren_gpu.py cases, Qwen3.5-2B at 4,096 rows): 15 of 15 right on ten requests for a change and five questions, the two from the report among them. No call was invalid, and a turn took 0.7 to 1.6 s. The reported conversation replayed on the same 2B: the status answer, then the coding answer in one round, then "Use two slots" in two.

0.7.1, 2026-10-11

Wren, the setup guide's assistant, is drawn as the gold DGX Spark box from the video "7 weeks with one DGX Spark" instead of a bird. The text does not change: only dashboard/ changed, and the engine's modules are the same bytes as in 0.7.0, so the disk caches keep their fingerprint.

• The box has the eight states the bird had, each with its own face. Idle, it smiles and blinks. It leans toward the message box as the user types, and question marks float above it as it thinks. Loading, it half closes its eyes inside the progress ring. Its mouth opens with the streamed text of an answer. To change a field it hops toward the page, and a small copy of the box lands on that field. An error brings a sweat drop and "?!", and asleep it closes its eyes under a drifting z.
• With reduced motion each state is a still face, and no two states look the same.
• This release swaps the dashboard's dist on a running engine, so /health reports 0.7.0 until the engine's next restart.

0.7.0, 2026-10-11

The dashboard has a setup guide at #/setup, with a small assistant called Wren beside it. The text does not change: the guide is not on the serving path, a chat request never reaches it, and the engine imports it only when a /v1/setup route is called. docs/setup.md describes all of it.

• The guide walks through an install in eight steps: the model, its overview with the measured table, the weights, the tools beside the model, the parallel requests, the methods, a review and the install. Next and Previous move between them, each step has its own link, and the progress stays in the browser's storage across a reload.
• A model the browser has no progress for starts from what the box runs now: the active profile and every knob as ops/serve.env resolves it. Review lists only what the user changed.
• A memory bar stays at the top. It shows the box's live free memory beside the selection's estimate, part by part, and the floor, and it turns red with a sentence when the selection does not fit. One formula in server/setup/estimate.py gives the numbers, and the page's copy must pass the same 47 cases in models/estimate-cases.json. exl3-4.5 with the recommended tools at 4 slots of 262,144 tokens comes to 67.35 GiB of a 106.63 GiB budget.
• The models the guide offers come from models/<id>/model.yaml. Qwen3.8-27B's file has the 7 weight profiles, the 0.6.2 table of 2026-10-10 (a test holds it equal to the run's summary file) and every tool with its memory, its serve.env keys and its default.
• Wren is Qwen3.5-2B on the engine's own forward with no caches, one sequence and thinking off. It answers from the docs, the manual, the registry and the packages that describe TandemLLM, and it reads the page and the box's free memory each turn. It can fill fields, tick tools, pick a profile and move between steps; the page runs each call after the server has checked it. It cannot finish, apply, reset, load or unload anything. On a serving box it loads only when asked, and it borrows the narrator's copy when that is loaded, which costs 0.03 GiB. Its own copy loads in 22 s and holds 3.62 GiB, it gives way to any request within one 150 ms slice, and it is the first thing given back when the KV pool needs memory.
• Finish starts an install job of fixed steps from server/setup/apply.py: wait until no request runs, write run/setup.env, switch the profile, and check that one chat answers. A failed switch or check brings back the old overlay and the old profile. Nothing a client sends becomes a command. The job's steps stream to the page, and the status responder keeps them coming while the engine restarts. ops/serve.env is never edited; it reads run/setup.env after the weight profile. The overlay leaves out a key serve.env leaves unset when its value is what the engine runs with anyway, so an install that changes nothing keeps the disk caches' fingerprint and the parked chats.
• Reset has five levels, in the System view and on the setup page: (1) the page's progress and Wren's chat, (2) the applied setup, (3) the caches and parked chats, (4) the weights of the profiles not in use, (5) the usage history and the conductor's data. Each lists what goes and its size, the destructive ones need a typed word, and the server refuses all of them while a request runs.
• Setup mode: install.sh --setup and ops/start.sh --setup start the dashboard and Wren alone, with no weights of Qwen's, for a box that has no configured engine yet. /health says "mode": "setup", /v1/models is an empty list, and every other route answers 503 setup_mode.
• Every POST under /v1/setup must be same-origin JSON. A text/plain body gets 415, and a cross-site Sec-Fetch-Site or an Origin that names another host gets 403. A page on another site cannot run apply or reset through a visitor's browser, even on a box with no admin password.
• VISION and RESPONSE_CACHE in serve.env now reach the engine (--vision off, --response-cache). Unset, they keep the old behaviour: vision on, the response cache off.
• The KV takes 104.25 KiB a token (106,752 bytes), not 106.5 KiB. The old figure was the kB value with the KiB unit. The memory and operations docs and the manual say so; engine/kvpool.py keeps the old wording, because its bytes are part of the disk tier's fingerprint, and changing them would sweep the saved chats.
• tools/exl3_fastrun.py --env-file reads serve.env without run/setup.env, so a benchmark row has the file's knobs, not the box's last setup.
• Measured on :8000 (EXL3 6.0, 2026-10-11 night):
• With Wren loaded on the narrator's copy, three greedy 512-token prose requests gave the same 411 tokens as three without it, and the first token's logprobs matched bit for bit. Decode ran at 148.6 tok/s with Wren and 147.5 without, and MemAvailable moved by 0.08 GiB.
• In setup mode, Wren's own copy loaded in 30 s and held 3.62 GiB.
• An install from exl3-6.0 to exl3-4.5 and back through the page took 243 s and 255 s. Both profiles kept their disk cache and their parked chats.
• The same prompt decoded at 32.17 tok/s against 0.6.2's 32.45 (one run each).

0.6.2, 2026-10-10

Decoding no longer stops while another slot prefills, and the dashboard's Perf tab reads the 0.6.x live feed correctly again. The text does not change: with another engine's 8- and 512-row forwards run at all 63 layer boundaries of a 24,576-row window, the prefill's logits, KV rows, recurrent state and convolution tails, and every logit of the other engine, matched running each alone bit for bit.

• An EXL3 prefill works through its prompt in 8,192-row windows, and each window was one GPU pass of about 10 s. Other slots got the GPU only between windows, so every decoding slot stopped for the whole pass (SRV-321). A window now offers the GPU turn between its layers, only when another slot is waiting. QSE_PARALLEL_LAYER_YIELD=off restores the old behaviour.
• Measured on :8000 (EXL3 4.5, one run each): three streamed chats with a 37k-token uncached prompt arriving 25 s later. Seconds with no decoded token during the prefill went from 42 of 147 (the longest 10 s) to 0 of 145. The prefill beside the decoders ran at 252 → 254 tok/s, a single request decoded at 42.5 → 42.1 tok/s, and the prompt alone prefilled at 809 → 792 tok/s (within single-run noise).
• Dashboard (VIS-76):
• The Now card's prefill bar fills again. A global tone rule in parallel.css had cleared the Live panel's colours.
• Between two window marks, the bar shows a hatched estimate at the measured rate and the ETA counts down. "measuring" replaces "warming up" until the second mark.
• The card and the prefill tile count the same total, including the harness's hidden tokens, on the same clock.
• The 5-minute charts lose the holes the 1.007 s sampler tick left, and say "none in 5 min" when nothing ran.
• A decode that stands still reads as "stalled behind a prefill" instead of a bare 0.
• The Parallel view counts lanes held for memory as waiting and uses the legend's words in the In flight tile.
• The Now line reads "4.9 of 17.2 tok/s" when several requests decode.
• The prefill's progress line is said only at a window's last chunk. Inside a window, a 47 s prefill read "about 4 min left".

0.6.1, 2026-10-10

The small models now give memory back when the KV pool runs short, and the dashboard's memory numbers count what is really in use. The text does not change: with the small models loaded, unloaded and loaded again, Qwen's 128 tokens, 26 blocks and prefill logits matched bit for bit.

• When the pool runs short, the order is: caches, then the shadow reader, then the narrator, then idle side conversations, then slot 0's idle conversation. Only after that does a request wait. An unloaded model is held out, so nothing loads it back on demand. It comes back through the residency planner once free memory has stayed above the floor plus its size plus 4 GiB for 60 s with nothing waiting. Laya stays loaded by default, because it decides routing and its fallback could change the text; QSE_ROLE_UNLOAD_LAYA=on puts it last. QSE_ROLES_DYNAMIC=off turns the whole feature off.
• In a live test, free memory went down to 24.9 GiB and a 31,687-token request arrived. The shadow reader went (1.82 GiB), then the narrator (3.81 GiB), then one idle side conversation. The request waited 11.5 s and finished, and both models came back 162 s and 238 s after the memory was released. Free memory never went below 21.7 GiB.
• The memory split counts the KV pool's committed pages, not the reserved address range of each slot. Before, it showed 16 GiB of KV and 10 GiB of drafter KV with 12.75 GiB in use. The bar now adds up to MemTotal minus MemAvailable.

0.6.0, 2026-10-10

One shared KV pool. Before this release, every slot reserved its whole window at load, so 4 slots fit only with 32,768-token side windows, and the narrator and Laya were refused. Now each slot reserves only addresses, up to 262,144 tokens, and memory pages are mapped as a conversation grows (QSE_KV_POOL=on). The text does not change: on 8k, 32k and 128k prompts the logits, every KV row and the GDN state match the static buffers bit for bit, and a greedy decode with the served drafters gives the same 258 tokens in the same 56 blocks.

Measured on :8000 (EXL3 4.5, 2026-10-10 night): 4 slots × 262,144, with Laya and the narrator loaded again. MemAvailable after warm-up is 58-63 GiB, against 26 GiB with the old 32k side windows. On the same 4-request load test, the aggregate went from 28.4-31.5 tok/s on 2 slots to 34.2-35.6 tok/s on 4, and the lowest MemAvailable from 23.7 to 48.5 GB. Three chats of 22k and one of 87k ran on four slots at once.

KV memory

• engine/kvpool.py reserves each slot's engine KV and both drafters' context KV as CUDA virtual memory, with the old buffers' exact shape. It maps 2 MiB pages just before rows are written: every forward pass, restore, pin, park, stash, disk tier, NVMe spill and side-slot carry. No attention kernel changed, and CUDA graphs keep their pointers.
• Pages past what a slot keeps go back when its request ends.
• Under pressure, caches give memory back first, then idle side conversations, then slot 0's idle conversation (to the NVMe). A 53.7k conversation came back from the NVMe in 1.2 s.
• Admission counts the pages a request could still commit, minus what it already holds.

Waiting instead of refusing

• A request that does not fit waits. Its stream says "waiting while memory is almost full… 1st in line" every 15 s and carries X-Tandem-Wait: memory, and the dashboard has a waiting_memory lane. A 95k-token request waited 111 s and then ran, with nothing refused. After the queue timeout, a non-streamed request gets a 503 memory_wait_timeout.

Parallel decoding

• Chats decoding on side slots no longer stall while another slot prefills a long prompt (QSE_BARRIER_IDLE_OWNER, on by default). In a same-hour A/B they went from 4.5 to 10.7-14.3 tok/s. The long prompt's first token came 18 % later (154.8 to 182 s), because the chats now get their share of the GPU.

0.5.1, 2026-10-10

Two dashboard numbers were wrong, and :8000 now runs four requests at once. The text does not change.

• The cache hit counts only the client's own prompt tokens that came from a cache. Rows the conductor adds are reported apart as hidden_tokens. The Parallel view had shown 117.2 % (112K cached of 95.3K prompt tokens).
• The live prefill rate counts only the rows the engine computed, so reused and restored rows no longer raise it. It had shown 17,892 tok/s on a prompt that ran at about 800.
• Four parallel requests on :8000 (QSE_PARALLEL=4): slot 0 keeps its 262,144-token window, and the three side slots get 32,768 tokens each. With 4 slots the memory guard leaves the narrator unloaded, and about 29 GB stays free after warm-up. A load test of 2 against 4 slots is still to come.

0.5.0, 2026-10-09

Weight profiles. The engine serves either our NVFP4 set or an EXL3 quant of Qwen3.8-27B, from 2.5 to 6.0 bits a weight, one at a time. ops/use-profile.sh <name> switches between them, and the model is still TandemLLM. The nvfp4 profile gives the same greedy text as 0.4.8 on the three prompts checked. An exl3-* profile serves other weights, so its text differs from NVFP4's. Either way, speculative decoding stays exact against whichever weights run.

Against BF16 on 8 held-out rows (16,376 positions), in this engine: NVFP4 has a KL of 0.0582 and agrees on the top token 93.35 % of the time. EXL3 4.5 has 0.0217 and 95.78 %, and reads 14.70 GB a decode step against NVFP4's 15.01.

Speed on EXL3 4.5, fast-run prompts, store off, same hour on 2026-10-09: greedy prose 41.3, chat 52.5, code 84.5 tok/s. The 50-prompt row runs 51.3 tok/s greedy and 50.9 sampled. TTFT with caches off is 8.8 s at 8k and 36.4 s at 32k.

Weights

• EXL3 linear layers behind QWEN38_EXL3: one GEMM kernel per bit width from 2 to 8, for 1 to 32 verify rows. The trellis decode matches exllamav3 bit for bit. With the flag off, the engine is the 0.4.8 engine.
• Both drafters run in EXL3 at 3 bits (QWEN38_DRAFT_EXL3), with a 4-bit EXL3 draft head (QWEN38_DRAFT_HEAD_EXL3). On the 50-prompt row they move EXL3 4.5 from 49.2 to 51.3 tok/s greedy.
• Profiles live in ops/profiles/*.env. ops/use-profile.sh fetches missing weights from the private Hugging Face repo after a disk check. A smoke request follows the start; if it fails, the old profile comes back. The crons start the active profile, and /health and the dashboard name it.

Prompt processing (prefill)

• Layer-major prefill windows decode each EXL3 weight slab once a window instead of once a 1,024-row chunk. With caches on, 32k goes from 52.2 to 42.2 s.
• A prefill MLP bridge feeds the prefill MLP's products straight into the down projection: 8k from 9.7 to 8.8 s, 32k from 39.1 to 36.3 s, caches off. Both are bit-identical.

Sampling

• A sampled round costs what a greedy one does: top-p reads each row's top 1,024 logits and the draws stay on the device, 97.4 to 89.1 ms a round.
• QWEN38_SAMPLE_NUCLEUS (on by default) draws inside each row's nucleus. The kept set is bit-identical, and the sampled-distribution gate passes.

Attention

• Verify attention runs three pipeline stages: 24 rows at 64k drop from 118.9 to 107.9 ms, bit-identical.

Tools

• tools/exl3: the Hugging Face Jobs pipeline that built the six EXL3 sizes.
• Also new: the lattice oracle, the sampled-tree replay and the drafter ranking-loss options. The ranking fine-tune (RND-37) gave no gain, and its tools stay for reuse.

0.4.8, 2026-10-08

Two Open WebUI chats beside each other, after a user report of 2026-10-08 (two chats that took turns on slot 0 with the side slot idle, and a chat read again cold, 60,814 tokens in 91 s, after its retrieval block moved). The text does not change: every resume here is held to the end anchor's gate.

Speed guard short gate on the 0.4.8-test engine code (2026-10-08 14:31, store off, one run a side, same hour): release 0.3.0 49.75, 0.4.8-test 50.64 tok/s mean (+1.8 %), 93.52 and 93.75 ms a round.

The router's notes

• A note stays live when the client changed the prompt before its message: besides the prefix hash it remembers the user message it closes (its ordinal) and the last 32 client tokens before it, and moves to that message's new end when the hash breaks (QWEN38_NOTE_ANCHOR). Open WebUI's retrieval block leaving an earlier message had dropped every note of the session and put the whole class note in again on every new message (5.8k tokens a message).
• A warm-up prefill of 2,600 tokens at start on slot 0 and on every side slot (QWEN38_WARM_PREFILL_TOKENS), so the first requests after a restart do not pay for the prefill kernels' first shapes.
• A new chat of at most 16,384 rendered tokens beside a conversation a side slot could never hold waits up to QWEN38_SIDE_NEW_WAIT_S (20 s) for a busy side slot before it takes slot 0 and sends that conversation to the NVMe: the eviction costs the long chat's next turn, the wait costs the new chat's first token only.

Parallel requests

• A side slot takes a request whose window leaves it at least 16,384 rows of output, with max_tokens cut to the room (QWEN38_SIDE_MIN_ROOM). 0.4.7 compared the window with the prompt plus the whole served limit of 32,768, so a chat past about 24,000 tokens could never take the 65,536-token side slot.
• A side slot has a resident prefix of its own (QWEN38_SIDE_RESIDENT_GB, QWEN38_SIDE_STASH_GB): the next turn of the conversation it holds resumes at the end anchor there (home), beside slot 0's conversation, which stays where it is; the router's question in between leaves it in place through the stash. The carry into slot 0 stays the fallback when that slot is busy for QWEN38_SIDE_HOME_WAIT_S or the prompt outgrew the window.
• A new conversation beside one a side slot could never hold takes an idle side slot before slot 0 (QWEN38_SIDE_NEW_FIRST), so the long conversation is not evicted for it.

Caches

• The end alignment's third pass: a block the client put into an earlier user message and moved to its last one (Open WebUI's retrieval block, on every request in the last user message) is served from the rows where the model saw it, token-aligned on the bare bytes, only when the same bytes are in the last message; a message the user edited is served as sent (QWEN38_END_ALIGN_MOVED). Without it the copy parted from the rows at the first message and the chat was read again cold on every new message.
• A long prompt the end anchor does not serve says why in the log and the journal ([cache] ... re-read ahead, extra.reread: the position and message the copy parts at, what the alignment refused) (QWEN38_REREAD_SAY_TOKENS).

0.4.7, 2026-10-08

The release after the 0.4.0 tree. It is 0.3.51-test (the 0.4.0 candidate) plus everything in 0.4.1-test to 0.4.7-test below, the manual, and the proxy documentation of the in-process DeepSeek relay. The 0.4.0 tree was never published; 0.4.7 replaces it. The text changes on purpose in three places: a regenerate under TandemLLM samples, Continue Response continues the partial answer, and the package documents change what the router attaches.

Speed guard short gate on the same engine code (0.4.7-test, 2026-10-08, store off, one run a side, same hour): release 0.3.0 50.38, 0.4.7 49.93 tok/s mean (-0.9 %), 93.73 and 93.74 ms a round.

Caches

• A long Open WebUI chat comes back from the NVMe after other chats ran: the restore offer asks the end alignment too, and a restore never evicts the entry it reads next. A 48k-token chat came back in 1.0 s with a TTFT of 2.3 s; a cold read takes about 70 s (0.4.2-test).
• The session spill is compressed losslessly: byte planes plus zstd level 1, 1.71x on the rows of a real 48,853-token entry, checked bit for bit. The spill budget is 64 GB, the disk tier 16 GB, and the NVMe gets a daily write budget of 200 GB (/health cache.disk_writes) (0.4.3-test).
• A conversation that ran on a side slot is carried into slot 0 with its rows and end state. A non-streamed answer keeps its end anchor. The park writes a conversation it gives up to the NVMe first (0.4.1-test).
• A deploy that only registers a knob keeps the disk's heads and conversations. The fingerprint reads engine/settings.py by the defaults of the knobs the engine reads, and it now reads the tools/ modules the engine imports. The first start adopts the old formula's files once (fp-equal.json). 0.4.6-test had swept 13 heads (12.4 GB) and 13 spilled conversations (29.6 GB) at its start (0.4.7-test).
• Open WebUI chats own their NVMe entries (X-OpenWebUI-Chat-Id), so one chat no longer takes another chat's entry: back in a chat after two others in 1.9 s, 6.9 s before (0.4.7-test).

Packages

• The first packages (the system set tandem-core, render-tools, eval-judge, and creative-coding, charts, voxel-worlds, explainer-video) and the everyday library: 34 tool-free packages plus everyday-core and reply-format (0.4.4-test to 0.4.7-test).
• Every prose class offers the whole everyday set; the router asks for a package once per class (package_sticky: class) and on an idle side slot, not on every new message and never over slot 0's conversation (0.4.6-test, 0.4.7-test).
• The everyday packages were measured against no package: 91 pairs, mean +1.19 of 5, judged by the served model, indicative only. Five ship off (schema.EVERYDAY_OFF). Everyday answers draft first, with a placeholder where only the user knows the fact, and never stop on a form (0.4.7-test).
• The testing package deletes only the scratch folder its own run made (.tandem/<session token>), and files-care never deletes what the conversation did not create (0.4.7-test).

Requests and answers

• Regenerate gives a different answer: it samples at temperature 0.7, top-p 0.95, top-k 20 with a fresh seed, and every other answer stays greedy (ledger schema 9, sampling) (0.4.4-test).
• Open WebUI's Continue Response continues the partial answer instead of writing a new one from the title (0.4.7-test).
• A memory note the store refuses is never shown as text, and a project note without a working directory lands at user scope (0.4.5-test).
• Laya stays out of the plan and the lines between tool calls; a refused Laya request says what was wrong and exits 2, a network failure still exits 3 (0.4.1-test, 0.4.5-test).

Dashboard, settings, tests

• The usage filter lists come from two new ledger indexes: the 365-day query is 845 ms cold and 110 ms warm (1,096 and 341 ms before). Seven server knobs are in the settings registry, and the CPU suite is green (0.4.6-test).
• The Deciders screen says why a model is not loaded (0.4.1-test).

Manual and documentation

• manual/: the TandemLLM manual as a LaTeX book, with the HTTP reference (manual/tools/http_ref.py) and the reference tables (manual/tools/gen_reference.py) generated from the code. .github/workflows/manual.yml builds the PDF only for a pushed v* tag and attaches it to that release.
• docs/proxy.md: the in-process relay's first live run and the serve.env lines. The proxy's start line names the config database's schema version.

0.4.7-test, 2026-10-08

0.4.6-test (b7be6e8) with the fixes for UX3's first regressions (FX15) and PKG3's measured package documents (v25-v27, router v21, below). The package texts go in as the box's packages v28 (tools/packages_migrate.py --put, edit 34, and the everyday skill copies), the router as v22 (package_sticky: class on the five prose classes). ~/rollback-0.4.7.sh restores packages v27, router v21 and the skill copies and returns the box to 0.4.6-test. Speed guard short gate (2026-10-08, store off, one run a side, same hour): release 0.3.0 50.38 -> 0.4.7-test 49.93 tok/s mean (-0.9 %), 93.73 -> 93.74 ms a round; no noise estimate by design.

A deploy that only registers a knob keeps the disk's heads and conversations (FX15)

• 0.4.6-test changed nothing under engine/ but engine/settings.py, where FX14 registered seven server knobs. The disk tier's fingerprint hashed every engine module's bytes, so that alone swept the 13 shared heads (12.4 GB) and the session spill's 13 conversations (29.6 GB) at its start: UX3's first Open WebUI chat read its 6.7k-token head cold (TTFT 14.3 s, 0.3.49 2.7 s), and the big chats kept on the NVMe went with it.
• The fingerprint now reads engine/settings.py by the defaults of the knobs the engine and the kernels read (a server knob's registration changes nothing), and it reads the tools/ modules the engine imports, which it never did (a kernel change kept rows another kernel wrote). The first start under the new formula adopts the files of the old one computed on the same tree and records that in fp-equal.json; later builds read them only under the same fingerprint. tests/test_fp_equivalents.py.

The package question once per class, on a side slot (FX15)

• Router v21 offers 34 everyday options on every prose class, and the package question (about 2,360 tokens) ran on every new user message: W02's "make it shorter" asked again and got the package the first note already held. The class policy's package_sticky: class, set on the five prose classes (router v22): a follow-up the class question keeps in the session's class takes the package picked for it (how: sticky); a class change or an answer of none asks again. The coding classes keep message (the old rule, also the default without a value; classifier.package_sticky sets the default): a coding session's next message is often another task with another skill (tests/test_note_dedup.py). tests/test_package_sticky.py.
• The router's questions take an idle side slot whenever slot 0 holds a conversation: the side-slot rule (_guest_of_slot0) no longer bounds a guest by a quarter of the conversation's length, a bound that belongs to a guest ON slot 0 (its stash). The package question over an 8.1k Open WebUI chat (limit 2,028) ran on slot 0, displaced the chat's end anchor (the next turn resumed at 0.96-0.98 and read 250-670 tokens again) and cost 3.3 s when it missed its own head there. tests/test_resident_guard.py.
• A side slot taken by the router's own question (or a side-lane job) notes no end to carry: the package question is longer than two chunks, and on build 1 its end was noted as a conversation, the next package question (same options) was carried into slot 0 over the chat, which was parked, and the note kept the next guests off the side slot for 10 minutes (live W01/W02).

Everyday answers draft first; no form, no stop to ask (FX15)

• W04: "Help me write a short cover letter ..." picked cv-cover-letter, whose step 1 said to ask for the experience "if the whole document depends on it", and everyday-core said "ask that one question and stop there". The model opened Open WebUI's blocking ask_user form; the message never finished and the user's next two messages queued in the client. everyday-core now says to take the likely value, state it in one line, put a [placeholder] where only the user knows the fact and give the whole result; never ask through a tool or a form, never stop a draft to ask. Eleven packages lose their "ask the one question" lines the same way (cv-cover-letter, checklist-builder, budget-plan, difficult-conversation, homework-help, tech-troubleshoot, translate, decision-helper, compare-options, weekly-schedule, parenting-help); the quiz turn and the practice conversation keep their question, which is the task. cv-cover-letter's max_tokens 950 -> 1050, everyday-core's 550 -> 650. tests/test_everyday_draft_first.py.

The testing package deletes only the scratch its own run made (FX15)

• PKG3's A01 run (opencode in a user's existing project) ran rm -rf /home/user/pago/.tandem, a folder with the user's own test kit, by the rule "a {tandem_scratch} left by an earlier run ... delete it before you start". The scratch is now the conversation's own folder (.tandem/<session token>, .tandem/session without one); the run marks it when it makes it and deletes it before "done" only with the mark; a leftover is reused and nothing is deleted at the start; files-care: never delete a file or folder this conversation did not create. tests/test_scratch_owner.py runs the commands themselves next to a user's .tandem.

Continue Response continues; Open WebUI chats own their NVMe entries (FX15; W17, SRV-285)

• W17: Open WebUI's "Continue Response" after Stop sends the conversation with the partial answer last. Every prompt was rendered with a generation prompt, so the partial was closed and a new answer started from the title (chatcmpl-9ea66357: 77 reasoning tokens, then 1,613 tokens; the saved answer held the story twice). A last assistant message with text and no tool calls is now continued (app.continues_final: the template's continue_final_message, the turn left open after its closed think block, no new reasoning); the body's continue_final_message or add_generation_prompt decide when given. The head is the earlier prompt's to the user turn's end, the note stays there, and a Continue is still not a regenerate. tests/test_continue_final.py.
• SRV-285: the caches' owner key read the body's ids and opencode's x-session-id, never X-OpenWebUI-Chat-Id, so every Open WebUI spill entry had no owner and a chat that shared a pasted document with another took the other's entry (F57: back in 6.9 s against 1.8 s on 0.4.1). The owner is now c:<chat id> when the body names none; an owned save never takes another owner's entry and an owner-less one never takes an owned one. tests/test_session_spill.py.

Earlier on main (PKG3, packages v25-v27 and router v21, documents only)

The everyday packages measured: five ship off; reply-format; write-message reworded (PKG3)

• The night A/B (2026-10-08, the stratified half of the 210 everyday eval prompts, n=1, the package attached alone against no package, same build and hour, judged by the served model at temperature 0 against each package's rubric, blind to the arm): 91 pairs, mean +1.19 of 5, 72 wins, 12 ties, 7 losses. Largest: gift-ideas +2.8, explain-simply +2.6, event-plan +2.6, fact-check +2.5, rewrite-text +2.47, tech-troubleshoot +2.4. A package costs 1.1-1.65k hidden tokens and 0.5-2.8 s of TTFT. The rubrics come from each package's own checks, so the judge favours the package's shape; a hand check of 16 pairs confirms the direction and finds the deltas too large. Indicative, not a release number.
• schema.EVERYDAY_OFF: creative-writing, language-practice, translate, social-post and summarize-text gained at most +0.3 (translate 5.0 both ways) and ship with enabled: false; they stay options of the package question, so a request that picks one gets nothing rather than a wrong package. health-info (+0.33) and security-scam (-0.10 on 2 judged pairs) stay on: the hand check shows the package's answers keep to the verdict and the steps and invent no numbers (without it an answer gave a made-up courier hotline).
• reply-format (new, about 270 tokens): an instruction about the reply itself ("Reply with just: OK", "answer in one word") picked write-message live (680 tokens on an OK); now reply-format takes it. write-message reads as a message from the user to someone they name. Over all 33+1 options in the live order: everyday 176/210 right (174 before), small talk picks a real package 3 times in 38 (6 before).

0.4.6-test, 2026-10-08

0.4.5-test (41a5b3d) with FX14's dashboard index seeks and registered knobs and PKG3's wide everyday package lists, deployed on the box as 0.4.6-test. ~/rollback-0.4.6.sh returns the box to 0.4.5-test (with the PKG3 schema cap of 01:28, router v20 kept or set back by ~/pkg3/wide/rollback-wide.sh).

Every prose class offers the whole everyday set (PKG3, measured 2026-10-08)

• Router v19 sent 106 of the 210 everyday eval prompts to chat (the plan class reads as software planning: 28 plans went to chat; research 49, writing 24), where the package question chose among 9. Over those 9 it picked the prompt's own package 38 times of 106 (none 51, wrong 17); over all 33 everyday packages 94 (none 6, wrong 6). Rewriting the plan, writing and research descriptions routed better (packages offered 122 -> 148-174 of 210) but moved the pagoda prompt from three_d to code_edit and cost 2 to 10 of the 341 labelled prompts, so the descriptions stay and the lists widen: writing, plan, research, summarize and chat list everyday-core and schema.EVERYDAY_ALL (33), with tandem-core as before. A class lists up to 48 packages (was 16).

The dashboard's filter lists by index seeks; a green CPU suite (FX14; SRV-281, QA SRV-282)

• The 365-day usage query over 500,000 rows took 771 ms on the Spark's fast core when SRV-29 set its 1 s bound, and 1,096 ms after VIS-60: dimensions(), run on every usage call, read every row twice (the internal-only clients, 165 ms; the proxy's users, 67 ms). The ledger gains two indexes in migrate() (requests_dim_kind on model, client_id, client_kind; a partial requests_user), built once at the next start, no row touched; the lists come from seeks. The cold query is 845 ms again, the warm one 110 ms (was 341).
• Settings: QWEN38_REGENERATE, QWEN38_REGEN_TEMPERATURE, QWEN38_REGEN_TOP_P, QWEN38_REGEN_TOP_K, QWEN38_MEMORY_PROXIES, QWEN38_TANDEM_URL and QWEN38_TANDEM_SCRATCH are in the settings registry (engine/settings.py), same defaults; tests/test_settings.py also finds a read split over two lines or held in a constant.
• Tests: the usage bound is the query's CPU time on the box's fastest cores, not the wall clock under load; test_proxy_gate uses a longer stream and repeats a run only when the run itself shows load.

0.4.5-test, 2026-10-08

0.4.4-test (dbdd4dd) plus three leftovers from the 0.4.x test runs, deployed on the box as 0.4.5-test; the package texts go in as the box's packages v24 (tools/packages_migrate.py --put, edits 30-33). ~/rollback-0.4.5.sh restores packages v23 and returns the box to 0.4.4-test. Raw and bench text are unchanged.

Live on the box (2026-10-08 00:26, 41a5b3d, about 4.5 min down; serve.env, router v19 and the DeepSeek relay kept): a dedicated test user in an Open WebUI-shaped chat asked for a project note, and it landed at user scope (rescoped: project) with no markup in the answer; W05 PASS (no_leak, follow-ups 1.00 at the end anchor, TTFT 1.4-2.0 s); O03 PASS with 0 of 13 visible text parts naming Laya (7 on 0.4.1-test), while the model wrote the new helper and used Laya twice; the helper against the live route: an empty text exit 2 "LAYA not asked", a duplicate label exit 2 with "choices: a label is listed twice (field: choices)" (0.4.4-test: exit 3 "LAYA unavailable (HTTP Error 422)"); W18 PASS (overlap 6.5 s on slots 0/1); DeepSeek flash 200.

A memory note the store refuses is never shown (SRV-277; QA SRV-278)

• In an Open WebUI chat (0.4.3-test, W05) the model wrote a <tandem_memory scope="project"> block. The chat has no working directory, so the store refused it (no_project), and the stream filter sent the refused block to the user as text, by design ("a block that is not taken goes out as the text it was").
• Now every closed block is hidden from the client, written or not: one over 2,000 characters, the fifth of an answer, one the schema or the store refused. The reason goes to the journal (memory_notes/write, the row's extra.memory.failed) and one log line, and the block's tokens are a hidden span like a written block's, so the next turn still resumes at the end anchor. Only text that is not a closed block goes out as written (an unclosed block, one still open after 4,000 characters, <tandem_memoryx).
• A project-scoped note in a conversation without a working directory (every Open WebUI chat, a script) lands at user scope (rescoped: project), since every client is the one user and a note is meant to be read back in the next conversation (SRV-173).

Laya stays out of the progress lines too (SRV-259; QA SRV-260)

• 0.4.1-test took Laya out of opencode's closing message, but 7 lines between tool calls in O03 still named it ("3. Ask Laya about the diff evidence", "Now the Laya verdicts on the key outputs"). The texts said only that the final message keeps Laya out.
• Packages edits 30-31: the texts that keep Laya out of the final message (test-lib, test-web, test-cli, test-api, test-data, testing, coding, tandem-harness, laya-testing) now say the plan and every line between tool calls reach the user too, and a check is named by what it checks. Guidance only; nothing filters the output.

A refused Laya request says what was wrong (SRV-279; QA SRV-280)

• In O03 the model piped its evidence into a heredoc script (tail ... | python3 - <<'EOF'), so the text was empty and the route answered 422. The helper caught every HTTP error as a network failure ("LAYA unavailable ... your code is not the cause"), and the model told the user "Laya is unavailable on this machine (transport error)".
• Packages edits 32-33: the helper sends no empty text ("LAYA not asked: the evidence text is empty ...", exit 2), and a 4xx other than 429 prints the route's message and field ("LAYA refused this request (HTTP 422): choices: a label is listed twice (field: choices). The request is wrong, not the network: fix it and ask again", exit 2). A refused connection, TLS, a timeout, 429 or 5xx is still "unavailable" (exit 3). laya-testing says a refused request is the model's to fix. The route's empty-text message says what to send; its param is unchanged.

0.4.4-test, 2026-10-07

0.4.3-test (2031c24) plus the packages merged on main (1a9ca7a, the section below) and Regenerate, deployed on the box as 0.4.4-test. ~/rollback-0.4.4.sh returns the box to 0.4.3-test. The text is the same for every request except a regenerate under TandemLLM, which now samples on purpose (the project's rule of 2026-10-07; docs/exactness.md).

Regenerate gives a different answer (SRV-241; QA SRV-242)

• Open WebUI's Regenerate sends the same conversation again with no sampling parameter. Under the greedy default it got the same answer word for word (UX2 W03), and on 0.3.49 the same five names in the same order.
• server/regenerate.py recognises a regenerate: under an active TandemLLM plan, the same session sends a conversation that ends with a user message and equals one already answered. The comparison skips system messages and reasoning fields, and the shapes come from Open WebUI 0.11.4's own source. X-Tandem-Regenerate: 1 also counts as one. Continue, Edit, a suggestion, a follow-up and tool rounds are never regenerates.
• A regenerate samples with temperature 0.7, top-p 0.95, top-k 20 and a fresh seed, through the engine's exact sampled path (the tree is kept under --sampled-tree det). The seed goes into the journal row. Everything else stays greedy, and so do raw and bench requests, side requests and compactions. A client that sends its own sampling parameters is served as it asked.
• Live on the box (23:48, Open WebUI 0.11.4, chat "[FX12] R12"). The first answer was greedy. The first regenerate had 4 of 5 names new and the second had 5 of 5 new against the first regenerate. The follow-up reworked the second regenerate's names. Both regenerates were cached 6,692 of 6,699 and the follow-up 6,829 of 6,846, resuming from the regenerated answer's rows; TTFT was 0.8, 0.7 and 3.0 s. Decode on the regenerates was 32.9 and 28.7 tok/s (3.5 and 3.1 tokens a block) against 44.3 and 35.4 tok/s (5.65 and 4.5) for the greedy turns of the same chat. The answers were short and n=2, so treat that only as a rough figure: sampling accepts fewer drafted tokens.
• Ledger schema 9: the sampling column (regenerate, client, null). The dashboard's speed sample leaves regenerate rows out, and the Requests API returns sampling. /health sampling.regenerate shows the settings and the counts. Off: --regenerate off.

The packages merged on main (PKG3, 1a9ca7a), shipped in 0.4.4-test

The first packages (OPS-39; QA OPS-40)

• Seven package folders under packages/: the system set tandem-core (the runtime's conventions, general on every class but summarize, 300 tokens), render-tools (how to use the render tool server of SRV-105, general on the classes that build, requires_tools: [render]) and eval-judge (the judge prompt and the rubric format, routed by no class); and the four first packages creative-coding, charts, voxel-worlds and explainer-video as library skills with their references, exemplars, eval sets (24, 20, 20, 12 prompts) and rubrics. docs/packages.md lists them; nothing is measured.
• Two tool capabilities for the tool fit (SRV-243): render (a tool with a string html argument) and render_video (scenes and cues), in requires_tools; a chat without the tool never sees render-tools, and explainer-video gives way to explainer-video-script (the scene script and cue list as the answer). A class may list 16 packages (was 12).
• tools/packages_first.py checks the folders against seed.json, INDEX.md and the router, measures the texts (the engine's tokenizer with --tokenizer), loads the eval sets and refuses private text; tests/test_packages_first.py runs it.
• Raw and bench text are unchanged; the router's class lists changed (tandem-core and render-tools on the classes above, the first packages as choices of the package question on drawing, three_d and design_ui).

The everyday library (OPS-79; QA OPS-80)

• 34 tool-free packages under packages/everyday/ (a letter, a CV, a trip or a meal plan, a budget, explain simply, compare options, a scam check, health information, a recipe, language practice and more) and everyday-core (the user's language, plain words, the whole result in the answer, one question at most, honest limits on health, money and safety), general on writing, plan, research, summarize and chat; the rest are the package question's choices on those classes (schema.EVERYDAY), tech-troubleshoot also on debug. 210 eval prompts in packages/everyday/evals/; nothing is measured yet.

Merging the two package sets (PKG3)

• The prose classes carry both: tandem-core, then the everyday packages (writing 11, plan 12, research 12); summarize the everyday ones only. tandem-core on chat is off until it is measured: schema.CHAT_CORE is the one-line switch, and on a box the router document's chat class lists it or not (live).
• The tool-fit note ("This chat can write files but has no shell") speaks of the shell and file tools only: a package that leaves because the chat lacks a tool of its own (render-tools without a render tool) leaves without the note. Before, every opencode turn on a class listing render-tools would have read that it has no shell.
• A first message under the accept bar is kept on its top class only when that class writes files (lists files-care, SRV-227), not when it lists any package: the prose classes name everyday packages now and still fall back to the default class (SRV-255's rule).

0.4.3-test, 2026-10-07

0.4.2-test (6fee8d3) with the spill compressed, the disk budgets of SRV-271's option A and a daily NVMe write budget, deployed on the box as 0.4.3-test. ~/rollback-0.4.3.sh returns the box to 0.4.2-test. The text is the same: every cache restores exact state, and the codec is lossless.

The spill entries compressed losslessly (SRV-275; QA SRV-276)

• A spill entry is the model's computed state, not text: 104 KiB a token (the target's 16 attention layers' K and V, 64 KiB, and the two drafters' K and V, 40 KiB, all BF16) and 0.77 GB of recurrent-state anchors an entry. The 182,000-row Open WebUI chat of 2026-10-07 was a 20.16 GB entry.
• New blocks are written as byte planes compressed with zstd level 1 (engine/spill.py, --session-spill-codec, SESSION_SPILL_CODEC, on): the planes are cut and joined on the device, the host compresses 8 MiB frames on 8 threads, one block while the one before it is written. On a real 48,853-token entry the rows go 5.20 -> 3.04 GB (1.71x; target 1.45x, drafters 2.39x), the entry 5.97 -> 3.81 GB; compress 4.5 GB/s, decompress 10.8 GB/s; the read of the compressed blocks plus their decompression took 0.93 s against 0.97 s for the raw O_DIRECT read. tools/spill_codec_check.py ran every block of two real entries through the codec: bit for bit. The block sums are checked on the device after every read, as before.
• Entries on the disk are read either way; a 0.4.2 entry written again keeps its unchanged blocks and compresses the new ones. The fingerprint skips the spill's files, so the deploy keeps every entry.
• Measured and not taken: plain zstd (1.27x target), zstd level 3, the fp32 recurrent state (1.17x), 2 anchors instead of 4. Lossy storage (FP8 KV, BF16 recurrent state) is reported in docs/caches.md and not implemented.
• To verify live: a long chat evicted and read back: the save line shows codec=zs1 and about 0.6x bytes, the restore line, cached 0.95 or more, TTFT within 1.5x of 0.4.2's restore.

The disk budgets: spill 64 GB, head cache 16 GB (SRV-271 option A)

• The session spill's budget 40 -> 64 GB (--session-spill-gb, now passed by ops/start.sh from SESSION_SPILL_GB, and settings.memory.session_spill.budget_gb live); the disk tier's 50 -> 16 GB (code and schema default; the box's settings document was set live before this deploy). The 50 GB free-space floor stays. Compressed, a 182k chat is about 12.1 GB: 64 GB holds about five.

A daily NVMe write budget (SRV-273; QA SRV-274)

• SMART on the box's 1 TB drive: 11.6 TB written, 1 % used. The disk tier and the spill wrote 137 to 282 GB a day (10-04 to 10-07). engine/disk_cache.py::WriteGuard counts both per local day (kept across restarts in writes.json); past --disk-write-budget-gb (200, DISK_WRITE_BUDGET_GB, settings.memory.disk_writes.budget_gb live) the tier writes nothing and the spill only a drain and evictions of 32,768 tokens or more; past 1.5x only the drain. Skips are logged ([disk-guard] skip) and counted; /health cache.disk_writes shows the day's GB, the level and 14 days.

0.4.2-test, 2026-10-07

0.4.1-test (585856d) and one fix, deployed on the box as 0.4.2-test. ~/rollback-0.4.2.sh returns the box to 0.4.1-test.

A long Open WebUI chat comes back from the NVMe after other chats ran (SRV-269; QA SRV-270)

• Live on 0.4.1-test: back in a 130,381-token Open WebUI chat after two other chats, the engine read the whole chat again cold (about 4 minutes at ~510 tokens a second) while it sat on the NVMe (182,053 rows with the reasoning, 20 GB, end=True). The restore's offer (Entry.offer) accepted the end anchor only through the hidden spans; Open WebUI's copy leaves every earlier answer's reasoning out, which only the end alignment (SRV-239) serves, so no entry was offered.
• The offer now asks the end alignment too, on the entry's own tokens, end anchor and saved regions (EndAlign.offers, SessionSpill.offer_align, wired in server/diskcache.py wire_end_align). A copy that differs in more than whitespace and the reasoning is offered a grid anchor at most, as before. The end anchor of a replayed or aligned copy is bounded by the prompt it is served as (the rows up to the end and the copy's tail), not by the copy's own length: the copy is shorter than the rows by what its client left out (130,381 tokens against an end at 182,053), and the first build of this fix (6bb9028) still stopped at a grid anchor there (live round trip: line 104,448 of 105,734).
• At a full disk budget, the restore's save of the resident conversation (before it reads the entry asked for) could evict that very entry, least recently used, and the read then failed into a cold prefill. No sweep evicts the entry a restore reads next; the resident conversation is then not written (refused.room).
• To verify live: a long Open WebUI-shaped chat, three other chats, back: [spill] restore in the log, cached 0.95 or more, TTFT under 15 s.

0.4.1-test, 2026-10-07

Branch fixes-041 from 0.3.51-test (3d4768e), deployed on the box as 0.4.1-test for a test on the box. The release 0.4.0 stays as it was. ~/rollback-0.4.1.sh returns the box to 0.3.51-test (a8ef6bf, the 0.4.0 candidate) with packages v21. The live checks are listed per item.

A conversation that ran on a side slot is carried into slot 0 (SRV-213; QA SRV-214)

• A new chat beside a long conversation starts as a guest on the side slot. When it outgrew the guest limit its next turn on slot 0 re-read everything past the disk tier's head: 8.7k tokens in 18.3 s (O07), cached 0.44 and 17.0 s to the first token beside an 80k conversation (W11).
• A side request's end is noted, and the conversation's next turn on slot 0 copies the side slot's rows and end state into slot 0 (about 104 KiB a token) after slot 0's conversation moves to the NVMe or the park; it resumes there as at any end anchor (cache_source side). QWEN38_SIDE_CARRY=off is 0.4.0.
• To verify live: W11 with an 80k conversation resident first, cached 0.95 or more and TTFT under 3 s on its follow-up.

A non-streamed answer keeps its end anchor (SRV-253; QA SRV-254)

• The session put ran before the end anchor. Under the store's 1 GiB entry cap (about 7,400 tokens) it cloned a snapshot, which settles the block's pending commit, then declined it as ahead of its key; the end anchor could no longer be cut back (5 of 45 non-streamed first turns on 0.3.50-test). The put now declines before it clones.
• To verify live: 20 non-streamed first turns with an Open WebUI-sized head, 0 not kept (ahead).

The per-entry cap stays; the park writes what it gives up (SRV-257; QA SRV-258)

• A 16,000-token snapshot is about 1.9 GB, the whole store, so the cap stays. A conversation between the cap and the spill's 16,384 tokens lived only in the park, which holds one; a third session dropped it. The park now writes it to the NVMe first (from 4,096 tokens; QWEN38_SPILL_PARK=off). The over-the-cap line is said once per conversation (165 of 245 requests printed it on 0.3.51-test).
• Live on 0.4.1-test, three Open WebUI chats in turn (A, B, C, A) showed a second gap: chats without an owner that share Open WebUI's ~6k-token head all saved into one NVMe entry, so C's save wrote over A's and A's restore failed its sums (the disk tier still served A at 0.964). A save now reuses an entry only when it matches it to within its last 4 chunks; another chat gets its own entry.

The Deciders screen says why a model is not loaded (VIS-70; QA VIS-71)

• MemAvailable 28.97 GiB less the engine's 8.25 GiB of growth leaves the shadow reader (6.5 GiB) at 14.22, under the 15 GiB floor. The growth is not counted twice: a request alone is admitted by the engine's own peak, not by MemAvailable. The screen now says "not loaded: kept free for requests" with the numbers in words and names the shadow reader.
• The Memory panel's "a load now" tile reads "kept free" without the warning colour, the sentence below it (live at 390 px it clipped and showed the warning).

opencode answers keep Laya out (SRV-259; QA SRV-260)

• Package edits 26-29: the test packages' evidence line keeps Laya's verdict for the model and says the final message names none; test-lib's check is "for you"; the done message is in the user's words; a failed Laya call is not news for the user. O03's answer had listed "Laya second check … passed (p=0.99, sure)".

Neutral comments (ENG-185; QA ENG-186)

• 300 comments and docstrings that named internal 0.3.x-test builds read in neutral words. No code changed.

0.3.51-test, 2026-10-07

0.3.50-test (77e4542) with the polish items of the normal-user scenario run (SRV-229, SRV-237, SRV-249, OPS-77; QA SRV-230, SRV-238, SRV-251, OPS-78) and two dashboard items from a user report on a phone (VIS-66, VIS-68; QA VIS-67, VIS-69). Raw and bench text are unchanged.

Live (builds 1-3: 103a8bc 08:59, 46a5574 10:04, a8ef6bf 10:15)

• W05: turn 0 (thinking off, 18,338 tokens) shows no "Thought for"; its status line ends "Read 18,338 tokens in 24 s"; follow-ups cached 1.00 six times. W18: "Thought for" 0 s and 2 s.
• Internal terms in final answers (Laya, p=, the kit, page check, hooks, .tandem, scratch, draw calls, fps ...): A01 21 (0.3.48) / 11 (0.3.49) -> 5; A04 22 -> 2 to 6; W10, W11, W12 0. opencode's O03 still names Laya: 39 (0.3.48), 43 (0.3.49), 26 (0.3.50) -> 24 (open).
• A01 (build 3): three_d kept below_build, the 3D skill and the reading line; the page renders a three.js pagoda garden (WebGL, 0 errors, no fps readout).
• Speed check (short gate, store off, one run each, the same hour): release 0.3.0 mean 50.50 / p50 47.45 / TTFT 428 ms, 0.3.51-test 51.38 / 48.48 / 423 ms (+1.7 %, +2.2 %, -1.1 %; no noise estimate by design).
• W11's follow-up re-read (cached 0.44) is SRV-213's shape (a new chat starts as a guest on the side slot beside an 80k resident), not a 0.3.51 change.

Open WebUI's "Thought for" is the thinking (SRV-237, SRV-249)

• Open WebUI 0.11.4 times a reasoning block from its first reasoning_content delta to its first content delta. The reading and waiting lines went out as reasoning, so a summary with thinking off that read 12,874 tokens showed "Thought for 14 seconds" (its block held only the two reading lines) and a one-line answer that waited 35.8 s showed "Thought for 34 seconds".
• For the client kinds in QWEN38_STATUS_CLIENTS (default open-webui; empty restores the profile's channel) every line said before the model's first generated token goes out as an Open WebUI status event ({"choices": [], "event": {"type": "status", ...}}), the line under the model's name that its tools use; at the first token one done event says what the wait was ("Read 12,874 tokens in 14 s", "Waited 31 s, then read ..."). The reasoning opens with the model's own first piece; a turn that does not think shows no "Thought for". A refusal after a wait closes the status line first.
• The waiting line's estimate reads the holder's own work (queue_wait.Live): the prefill it still has to read at its live rate, plus the 75th percentile of what earlier requests of its kind (thinking or not) wrote past the tokens it has written, at its own decode rate, capped by its max_tokens. Replayed over the box's ledger (3,716 client requests, the waiter arriving at 10 to 90 % of each hold): for waits of 10 to 60 s the old estimate was a median 0.52 of the real wait (78 % too low), the new one 1.06 (42 %); over a minute 0.17 against 0.59. Waits under 10 s now read about twice their length. Without a view of the holder the hold-time median stays.

Final answers for the user (SRV-229)

• A01's answer to "create a pagoda garden" quoted "Laya says fix_first (p=0.596, sure: false)", "the kit's page check", "renderer.info: 56 draw calls" and the removed test hook, and the page kept an "fps · calls · tris" line. The texts asked for it ("quote Laya's verdicts", "say in your final message that you removed it", "Report: ... Laya's verdicts", "fps reported").
• Package edits 15-23 (tools/packages_migrate.py, applied to the shipped texts, a box's packages document and its skill files): the final message says what was built, where, how to open and use it, one plain sentence on how it was checked, what is untested; Laya, its labels and probabilities, the page check, hooks, scratch, draw calls and fps stay out unless the user asks; anything added only for the check (an fps or stats readout, a debug panel) goes before "done" or behind ?debug. The S5 and E2 notes ask for the same in plain words (tools/monitor_migrate.py, ops/supervisor.yaml).

Docs (OPS-77)

• The state store's served budget is 2 GiB with 1 GiB a snapshot everywhere (two places said 8 GiB); the guest rule names both conditions (at most 8,192 tokens and under a quarter of the resident prompt); 49.89 tok/s is the mean of the requests' own rates, not 4.58 tokens / 93.4 ms (49.0); the spill's re-read of 58,278 tokens costs about 83 s at the ~700 tok/s the ledger shows for cold prefills that long (it said 57 s). tests/test_docs_defaults.py checks the documented budgets against the flags and ops/serve.env; parallel-requests.md says which settings the reference box changes from the shipped serve.env.

A build request that the router is unsure about (SRV-255)

• The request "Create a new folder and create there a pagoda garden" went to chat on router v17: the chat class's wider description (SRV-235) moved three_d from 0.464 to 0.378, under the 0.45 bar, and a first message under the bar always took chat. No 3D skill, an ASCII pagoda in a file, and the argument loop guard stopped the write twice (it was right: one 28-character row 151 times).
• classifier.below_accept (shipped: build, margin 0.1; router v18 on the box): a first message under the bar keeps its top class when that class builds something (its policy names packages) and leads the second by 0.1. 341 labelled prompts on v17's answers: 307 -> 309 right, none newly wrong; "Reply with just: OK" stays chat at 0.999.

The dashboard on a phone, and the speed check (VIS-66, VIS-68)

• An editor's save bar no longer floats over the conductor's cards when nothing is unsaved (Deciders at 390 px hid "a load now: refused"; Monitor too, also at 768 px, Chromium and WebKit): idle bars sit in the page's flow; unsaved edits keep Save above the tab bar. e2e "an idle save bar covers nothing" (6 screens, 390 and 768 px).
• The speed guard is no warning banner on the Conductor view any more. System → Engine and the Dev tab's model card carry one neutral "speed check" line ("passed for · ", "not run for this build"); a one-run short gate whose mean and p50 dropped no more than 5 % reads as passed; Compare keeps the numbers in a muted line. A release tree ships its report as release/speed-guard.json (tools/speed_guard.py ship), listed by GET /v1/conductor/evals.

0.3.50-test, 2026-10-07

0.3.49-test (e6970f5) with seven fixes from the normal-user scenario run and FX7's measurements (SRV-243, SRV-239, SRV-245 with SRV-219, SRV-247, SRV-233, SRV-250, SRV-235; QA SRV-244, SRV-240, SRV-246, SRV-220, SRV-248, SRV-234, SRV-236). Raw and bench text are unchanged. Under a conductor profile an API client that names no reasoning_format now gets reasoning_content (a changed default), and the follow-up of an answer that thought is served with that answer's reasoning.

A chat gets the packages its tools can carry out (SRV-243)

• A plain Open WebUI chat (its tools: search_web, fetch_url) asking for a landing page got the testing and Laya packages, which tell the model to run a shell and write files. It called terminal and write_file ('Tool not found'), fetched the library's address with fetch_url, wrote the page into a failed call and printed it again: 518 s on 0.3.48-test; a snake game 423 s, a 3D garden 522 s.
• The router reads what the request's tools can do off their schemas (a shell: a command argument; a file write: a path and a content argument). A package names what its steps need in requires_tools (absent: a shell when its text calls TandemLLM through {tandem_url}); one the tools cannot carry out stays out or gives way to its fallback (coding to the new coding-answer; on the reference box web-design to web-design-answer, packages v18), and the note says which tools the chat has and to deliver the files in the answer. The harness, library and memory routes need a shell too. Decision rows: extra.tools. QWEN38_TOOL_FIT=off keeps every package.
• Live on 0.3.50-test (n = 1 each): W10 one request, no tool call, 294 s (315 s with web-design-answer); W11 195 s and its fix turn 116 s; W12 248 s; notes of 404 to 2,321 tokens instead of 4,562 to 7,590. What is left is the model's own work: 7,000 to 10,000 reasoning tokens and an 8,000 to 9,000-token page.

The follow-up of an answer that thought resumes at its end (SRV-239)

• Open WebUI sends an answer back without its reasoning; the Qwen3.8 template renders it as an empty <think> block, so the client's copy parted from the engine's rows right after <think> and the turn resumed at the grid line below. Six of six Open WebUI chats whose first answer thought re-read it on the first follow-up (0.71 to 0.89, TTFT 2.5 to 5.0 s), and a long chat resumed at the grid on every turn after one that thought (W05: 0.985 falling to 0.947).
• The end alignment remembers where each generation's reasoning ends and, when the copy does not align as the rows are, aligns it with the reasoning taken out; the prompt is served with the rows' tokens to the end anchor (the reasoning the engine wrote included, as the template renders it for a client that sends it) and resumes there. QWEN38_END_ALIGN_REASONING=off is the old rule.
• A/B on 0.3.49-test, one build, the follow-up without the reasoning against with it as reasoning_content (n = 2 each): 0 tokens reused against 309 and 319. On 0.3.50-test with an Open WebUI-sized head: without the reasoning 6,324 to 6,340 client tokens reused at the end anchor, TTFT 112 to 131 ms, like the copy with it. Live first follow-ups: W02 7,300 of 7,315 (TTFT 0.17 s; 0.83 and 3.96 s on 0.3.48), W03 6,880 of 6,897, W04 7,018 of 7,044, W15 6,774 of 6,990, W16 7,095 of 7,116, W17 after Stop and Continue 8,276 of 8,306 (TTFT 1.0 s; 5.0 s before).

API clients get a clean answer (SRV-245, SRV-219, SRV-247)

• Every client kind but the Hermes agent gets reasoning_content when it names no format (QWEN38_CLIENT_REASONING=*=reasoning_content,hermes=tags): an OpenAI SDK printed the thinking as its answer, and opencode with the installer's config showed literal <think> text. reasoning_format: tags keeps the tags. The answer starts with its first character.
• stop strings apply to the answer only: stop: ["\n\n"] ended a response inside the thinking with no answer; now it returns the first paragraph (P04). State lines never go into a plain client's content.
• A request that offered no tools never gets tool_calls; a call in the answer is dropped (extra.calls_refused), and a request without a shell gets the memory index without its routes. P08: "What is my UX1 pet?" answers "I don't have any note about a UX1 pet", finish stop (0.3.48-test: a call to a tool named tool, finish tool_calls).

A trivial instruction stays cheap (SRV-235)

• "Reply with just: OK. Do not use any tools." in a new opencode session went to code_edit (0.80) at xhigh with the 4.9k-token coding package: 9.3 s to "OK". Unquoted, the class question put chat first at 0.41, under the 0.45 accept, so a first message fell back to the profile's default class (code_edit); quoted, as the scenario sent it, Qwen said code_edit 0.80.
• The chat class now covers an instruction about the reply itself (answer in one word, say OK, confirm, stop) that asks for no work on code, files, commands or git (router v17 on the box, the shipped router). On the labelled sets (308 prompts) and 20 short opencode messages (tests/fixtures/conductor/short_prompts.jsonl, 10 about the reply, 10 coding) the live class question went from 286 to 297 of 328 right (the prose set 18 to 27, the task set 198 to 200, long 51 to 51, short 19 to 19); two answers got worse ("Commit the changes with a short message" is chat, "What does ASAP stand for?" code_edit). Live in opencode: the OK message is chat at 0.999, no note, TTFT 2.3 s; "fix the failing test in textstats.py" stays debug at 0.999 with the coding package.

cached_tokens counts the client's tokens (SRV-233)

• The hidden rows inside the cached span counted as client tokens: an opencode turn that re-read 3,827 of its 24,706 client tokens reported 24,706 cached. The server now tracks where the hidden tokens sit and reports the client's tokens below the resume point (20,879 for that turn). Ledger schema 8 adds engine_cached, forwarded_tokens and wait_mem_ms.

The parallel admission sizes a request by what it needs (SRV-250)

• A request counted the workspace floor plus a whole state-store entry (2.1 GiB for 576 tokens) against 6.4 to 8.4 GiB of cache budgets: at the box's idle MemAvailable a 576-token title request waited 63 and 83 s beside a 9,000-token one, booked as prefill. Now its need is its own prefill workspace, and the caches' growth, the requests in flight and its own together are capped by the engine's room to --mem-peak-gib (every cache asks the memory policy before it grows and every prefill passes its gate); the wait is queue time (wait_mem_ms) with a waiting line, and the alone path says so.

0.3.49-test, 2026-10-07

0.3.48-test (c16b27c) with four fixes from the normal-user scenario run (SRV-223, SRV-225, SRV-227, SRV-231; QA SRV-224, SRV-226, SRV-228, SRV-232). Raw and bench text are unchanged.

A second chat no longer waits for the first

• The cause (SRV-223): with --parallel 2 a request "continued" slot 0's conversation when it shared all of its rows but the last chunk. Open WebUI's system prompt and tool block (about 6,700 tokens) are the same for every chat, so beside a conversation shorter than that head plus one chunk every new chat of the same client counted as its next turn and waited for slot 0. W18 on 0.3.48-test: '17 x 23' in a second tab shared 6,675 of slot 0's 7,096 rows and waited 35.8 s while slot 1 was idle.
• Both continuation tests (the client view and the resident rows) now also require the shared part to reach past the end of the request's first user message (_conv_start, the chat template's <|im_start|>user ... <|im_end|> marks). QWEN38_SIDE_CONV=off is the old rule; the [parallel] line says new conversation (shared head N).
• Live on 0.3.49-test (W18, n = 1): the second tab ran on slot 1 inside the first tab's 67 s essay, queue 1.29 s (the side slot was finishing the router's 412-token question), TTFT 2.6 s, 6,144 of its 6,695 tokens from the disk tier, answer 391; Open WebUI showed "Thought for less than a second".

A side slot reads the disk tier (SRV-231)

• A side slot had no disk tier, so a new chat there prefilled its client's whole head (0.3.48-test: Open WebUI 6,687 tokens, 9.0 s TTFT; an opencode first turn 19.1 s on slot 1 against 10.7 s on slot 0). It now reads slot 0's tier through a read-only view (server/diskcache.py SideReader): find and restore into the side engine's own buffers; no line counted, no write, no fence of slot 0's write-behind jobs. /health parallel.side_disk; QWEN38_SIDE_DISK=off is the old slot. A/B on 0.3.49-test with the tier's mode switched (W18 tab B, a new Open WebUI chat on slot 1, n = 2 each): TTFT 9.6/9.8 s off, 1.9/1.9 s on. Across builds (O03's opencode first turn on slot 1, n = 1): 19.1 s with 12,570 tokens forwarded on 0.3.48-test, 10.5 s with 5,603 on 0.3.49-test (what is left is mostly the router's 5,310-token note, which no cache holds on a side slot). tests/test_disk_cache.py: a restore into an engine with a shorter window equals its cold prefill bit for bit.

One conductor session per Open WebUI chat (SRV-225)

• Open WebUI sends no chat id by default, and the session key fell back to a hash of the head up to the first user message: every chat that opened with the same sentence was one session (A01: a new chat's first row counted 27 turns and 21,216 session tokens of earlier chats and replayed their 4 notes).
• The engine reads X-OpenWebUI-Chat-Id as a conversation id. On the reference box Open WebUI's TandemLLM connection sends it as a connection header ({"X-OpenWebUI-Chat-Id": "{{CHAT_ID}}"}), not through ENABLE_FORWARD_USER_INFO_HEADERS, which would send the user's name and email to every connection (docs/operations.md, Open WebUI).
• Without an id, a head holds a list of conversations: a conversation's first turn starts its own session (h:<hash>#n) when the newest one of that head has had a turn; a later turn finds its own by a hash of its whole first turn as the client sends it back. Decision rows carry extra.session (id, new, continued, alias). QWEN38_SESSION_GENERATIONS=off is the old rule.

A new folder never replaces the user's files (SRV-227)

• "Create a new folder and create there a pagoda garden" ran mkdir -p pagoda-garden on an existing folder and wrote index.html over the user's page without reading it or saying so. The tools run in the client, so the fix is what the model is told: the general package files-care (a new folder or file gets a free name, a file is read before it changes, a replaced file is copied first, the answer says so) on every class that writes files (code_edit, debug, design_ui, drawing, three_d); packages v16 and router v16 on the box.

0.3.48-test, 2026-10-07

0.3.47-test with one router fix (SRV-205; QA SRV-206). Raw and bench text are unchanged; under a conductor plan a three_d turn now always carries the 3D skill, and an unsure visual class adds one sentence to the note.

A routed class brings its own skill

• A class's policy takes core: packages that go in whole on every turn of the class, like a general package but for that class alone (journal extra.package_core). On 2026-10-06 "Create a new folder and create there a pagoda garden" went to three_d at 0.464; the 3D skill was only one of the package question's options beside none, the question answered none at 0.747, and the model built a flat SVG page. The box's router v15 makes threejs-scene three_d's core; the shipped router also makes coding the core of code_edit and debug (the shipped packages document has no general flag on it, so a new box offered it to the question).
• A router may give the package question's Qwen stage its own prompt, classifier.package_prompt, with the class the turn was routed to ({class}). The default stays the class question's prompt with the packages as the classes: three prompts that name the class (n = 1, 51 prompts) cost 8 to 27 test-skill picks on code_edit and debug and still left three of four unsure 3D prompts under the 0.6 accept. The library no longer picks a package the turn's note already carries.
• The reading line (classifier.reading, under the profile's classify decision): a class a model stage picked on the turn the class starts, below 0.8, with another class at 0.15 or more, both with a reading phrase (three_d "a 3D three.js scene", design_ui "a flat web page", drawing "a 2D drawing"), tells the model to build the picked kind and to open its answer with one short line such as "Building it as a 3D three.js scene. Say if you wanted a flat web page.", without asking first. mode: shadow journals it only.
• Measured with tools/class_core_eval.py on the live class question (router v14, n = 1): on the labelled sets (308 prompts) the class answers and every class's package picks are the same as before, and the three_d turns that get the 3D skill go from 6 of 22 to 22 of 22; on 13 ambiguous prompts (tests/fixtures/conductor/ambiguous_prompts.jsonl) from 3 of 7 to 7 of 7. The reading line fires on 7 of the 13 ambiguous prompts and on none of the 308 labelled ones.

0.3.47-test, 2026-10-06

0.3.46-test (6f985a4) with the fix for a new user message that read a long conversation again (SRV-207, SRV-209; QA SRV-208, SRV-210). Raw and bench text are unchanged.

A short request no longer costs the conversation slot 0 holds

• The cause (SRV-207, one build, replayed on the live engine, n = 1 each): with --parallel 2 the first request's residency document set the memory ceiling and peak back to their configured 70 and 77 GiB, so the 7.65 GiB the side slot took at the load came out of the caches' room. Every cache was then refused (one evening on 0.3.46-test: 131 anchors, 2 guest stashes, a park, a pin), and the router's own questions before a new user message (its class and its package, 410 and 169 tokens), which run as guests of the conversation, cut its rows at their start when their stash was refused. A 32,875-token conversation and two new user messages: 0 % of the prompt from the cache at the 70 GiB ceiling (67.5 s and 70.7 s), 99.9 % and 99.3 % at 78 GiB (6.4 s and 9.4 s). In the box's logs of that day, the two processes started with one sequence (0.3.42-test) kept every guest's stash (318 guests, none cut); the cut appears only in processes with a side slot.
• The side slots' bytes are kept apart in the memory policy (Budget.fixed, /health cache.memory.fixed) and added to every value the ceiling and peak get, from the flags or a residency document.
• With parallel requests, a guest of slot 0's conversation (short, sharing less than one chunk with it, while slot 0 holds two chunks or more) takes an idle side slot first and leaves slot 0's rows alone (QWEN38_SIDE_GUESTS, default on; off is 0.3.46-test's slot 0 with the stash). The router's questions and a client's title, tags and follow-up tasks are such requests.
• A guest's stash holds the rows the guest can write (its prompt, its max_tokens, 2,048 rows for the monitor's hidden tokens, two 512-row margins), not every row its 1 GiB budget holds: a 410-token question asked the memory policy for 1 GiB.

A new task skill no longer brings the whole note again (third build, SRV-211)

• Live on the second build (17fcb69), Open WebUI agent chat: every user message of the coding class whose task skill differed from the last one's (test-lib, none, test-cli) brought the router's whole note again, the five general packages included (about 4,800 tokens each time); the dedup key covered the whole block. The third copy pushed the replayed notes over injection.max_replay_tokens (16,000), the oldest note went, the prompt changed at its position (12,292) and the turn read 35,442 tokens again (49 s) while the client's own tokens had not changed.
• A class's note now carries only the parts (the effort sentence, each package, the rules) that no live note of the session holds with the same text; every part live, no note (QWEN38_NOTE_DEDUP, parts; block is the old rule). The proxy's harness follows the same rule.
• Replayed with one conversation shape on the live engine (n = 1 each; a 20,000-token document in the second message, then four code messages that pick four task skills): on the second build the fifth message read everything again (cached share 0.000, 60.7 s) and messages three to six took 115.6 s; on the third build (c2af3fe) every one of them read from the cache (cached share 1.000) and they took 17.3 s, the replayed notes 6,040 tokens instead of 15,406.

Open WebUI's working directory

• Open WebUI appends the terminal's live working directory to run_command's description, in the system prompt's tool block (SRV-209). The first cd of an agent changed the prompt about 4,800 tokens in, and the next user message read the conversation again (live: 41,749 tokens, 56 s, of which the cache could have served about 4,800). Under a conductor plan that is on, the sentence moves into a router note at the turn where the directory changed and replays there (QWEN38_CLIENT_CWD, default note; off: the tools as sent).

0.3.46-test, 2026-10-06

0.3.45-test (04ada2a) with two fixes (SRV-197, SRV-199; QA SRV-198, SRV-200). Raw and bench text are unchanged.

Parallel requests keep an agent's cache

• A turn that continues slot 0's conversation waits for slot 0 (SRV-197), judged on the client's rendered tokens against the last prompt whose rows stayed resident. The old test compared the request with the resident rows, which under a conductor plan hold the router's notes and the end of the last generation: no agent turn ever counted as a continuation, and a turn that met slot 0 held for a moment by the shadow reader took the side slot and read its whole conversation again. QWEN38_SIDE_CONTINUE=resident restores the old test.
• While only TandemLLM's own work holds slot 0 (the shadow reader, a summary being prepared, the memory writer, the router's question), a request waits for slot 0, up to 15 s, instead of taking a side slot: that work gives way at its next safe point, and a side slot has none of slot 0's caches. A side slot serves a second client's request.
• Two long agent sessions at once mostly take turns on slot 0, each resuming its own conversation: measured with two opencode sessions (n = 1 each), both done in 301 s with cached shares 0.90 and 1.00, against 490 s when one of them was sent to the side slot whenever slot 0 was busy (that session at 0.26: a side slot keeps no conversation) and 193 s one after the other. A follow-up ticket covers side slots that keep their conversation.
• The [parallel] log line says which slot a request took, on which numbers and who held slot 0; [req] has slot=; the usage ledger moves to schema 7 with a slot column (null without parallel requests), also in the dashboard's request rows.

A call written inside the thinking

• E1 case C (SRV-199): a turn that ends with complete tool calls inside its reasoning and no answer gets those calls, the model's own tokens, as its answer, when each parses and names an offered tool; otherwise E1's hidden close or nudge, and GIVE_UP's sentence after two tries. E1 now acts in the chat and general profiles too.

0.3.45-test, 2026-10-06

0.3.44-test (d33967a) with the two follow-ups of the second self-test round (SRV-193, SRV-195; QA SRV-194, SRV-196). Raw and bench text are unchanged; under a conductor plan Open WebUI's reasoning moves from <think> tags in the content to reasoning_content.

Open WebUI

• Open WebUI gets reasoning_content when it names no reasoning_format (SRV-193, QWEN38_CLIENT_REASONING, default open-webui=reasoning_content; empty restores tags; a request's own format wins; the raw path keeps the server's). Open WebUI 0.11.3 forwards the stream piece that carries </think> to its live view raw, as the text of a new message, until the round's stream ends: in a long agent chat 15 of 70 page scans showed a literal </think>. The template's blank lines go out as one content piece where the reasoning ends, so every tool round's block is closed and timed ("Thought for N seconds"), also when the round goes straight to a tool call.

Testing a page

• laya_drive.py page URL --step ..., the kit's one-command check of a page (SRV-195): its own headless Chrome, real clicks and keys, values read back, screenshots at 1280 and 390 px, console errors, uncaught exceptions and failed loads, Laya's verdict, Chrome closed on every path, about 5 s. opencode had found no node, npx or Chrome on its PATH, only Firefox, and spent seven minutes on a Marionette driver of its own after the page was written.
• The kit starts Chrome with --password-store=basic --use-mock-keychain: every start (playtests too) had waited about 25 s for the desktop keyring at the first network request.
• The testing packages (tools/packages_migrate.py, edits 9 to 11): a page is checked with the page check, never with a driver of one's own; NO BROWSER or two failed checks mean reading the code and saying so; after the deliverable exists: check, fix, check, then the report; a one-screen tool (a timer, a form, a calculator) needs no Laya playtest.
• S5 gains a fourth arm, verify (policy 7): a build turn whose steps since its last product write wrote only scratch files, two of them test tools (scripts or pages, not logs or data), 90 s or more after that write -> "your last N tool steps changed only test tooling ... Stop building test tools ... run one check ... then answer". On the recorded runs: FX2's pomodoro gets the note at 347 s (the run went on to its 600 s limit), four clean opencode runs get none. tools/monitor_migrate.py adds the new keys to a box's stored S5 and keeps its own values.

0.3.44-test, 2026-10-06

0.3.43-test (a3bf8a5) with parallel requests made workable beside the small models, and no in-flight limit from the proxy on other backends (OPS-69, QA OPS-70). A design decision: parallel requests at 4 or 2, with Laya and the narrator (the describer) always loaded, so three models. With --parallel 1 (the shipped default) a request is served as before, and the text is unchanged.

Parallel requests

• --side-graphs off (QSE_SIDE_GRAPHS): a side slot verifies eager when it decodes alone and holds no CUDA graph pool. A joint pass of two decoding slots is eager either way, so the slot loses its graphs only for the stretches it decodes alone, and it costs its window plus about 1.5 GiB instead of 7.5. The build's estimate follows the switch; QSE_SIDE_SLOT_CONST_GB overrides it. /health's parallel.slots[].graphs says which slots hold graphs. The default stays on.
• docs/parallel-requests.md "Beside the small models": the memory plan of a 128 GB box with Laya, the narrator and two sequences, why the side window is 65,536 and not 8,192 (the conductor's margin and the clients' max_tokens), the planner's claims, and what Laya and the narrator do while two requests run.
• The reference box serves two requests at once from this version (its serve.env, not the shipped one).

The proxy

• No in-flight limit from us: a relayed backend (another TandemLLM, vLLM, SGLang, an OpenAI-compatible server, a cloud API) takes as many requests at once as clients send, unless its max_inflight is set (1 to 4,096; the form's field empty, null, or TANDEM_BACKEND_<ID>_MAX_INFLIGHT=none is no limit). Until now every backend got 64 without asking and a 65th request got a 503. The config file moves to schema 2, which turns a stored 64 into none and keeps any other number.
• The Backends screen shows the field empty with "no limit", the card without "/64", and a line for this engine's own settings: how many requests it decodes at once, its decision models and its narrator. The parallel setting is the engine's, not a backend's.

0.3.43-test, 2026-10-06

0.3.42-test (bfe19ef) with the fixes from the second user-level self-test the same day (UT2: about 26 scenarios as the owner uses the engine, Open WebUI in a browser at 1440 and 390 px, opencode, the OpenAI SDK, the dashboard), and the identity note the owner asked for. QA tickets: SRV-180, 182, 186, 188, 190, 192, OPS-68, VIS-59, VIS-61.

Agent turns that end with an answer

• S5, a new session task (turn_budget, SRV-179 and SRV-181), on every client and project. repeat: the model's visible text between its tool calls states the same problem three times among its last eight problem leads (shared word 3-grams, a number in common) -> "you have now stated the same problem 3 times ... Stop tuning it ... Deliver what works ... Do not change the product only to make your own test pass". first_write: a build turn (code_edit, design_ui, drawing, three_d) three minutes and five tool steps in with nothing the user asked for written -> "Write the deliverable now, before any more setup". wall_clock: an Open WebUI turn with tool calls for 12 minutes and no product write for 4, or for 18 minutes in any case -> "the user is waiting in a chat for your answer. Finish now", then every 4 minutes the shorter "Answer the user now". Replayed on the UT2 Mario chat (43 minutes, no answer): the notes come at 3.1, 18.0 and 36.0 minutes; on four clean opencode runs it adds nothing. tools/monitor_migrate.py adds it to a box's stored monitor document.
• The testing packages (tools/packages_migrate.py, edits 5 to 7): the deliverable before any test tooling, a verification tool that fails twice is dropped with one sentence, a server runs in the background with a time limit and its pid in scratch and is stopped before "done", a scratch directory a cut run left is cleaned first, what is enough verification, never change the product for one's own test, the final report; Laya's playtest is a check and not a deliverable, a spot only the bot fails is not a bug, two runs ending at the same spot end it.

Chat clients

• A memory write's line (› remembered: ..., proposed, forgot) goes into the content after the answer in every state-line channel but opencode's inline (SRV-185): in tags a line after </think> had been dropped, so Open WebUI and the SDK never showed a write.
• A bare </think> line the model writes again in its answer is not sent (SRV-187); Open WebUI showed it as text between two tool rows. Code, inline code, tool calls, memory blocks and a block the answer opened itself keep theirs; the raw path is untouched.
• Asked who or what it is (EN, DE, FR, ES, IT, NL), the model names Qwen3.8-27B and says it runs in TandemLLM (SRV-191): one hidden note on the turn that asks, replayed at its place on later turns, under every profile but raw and bench. QWEN38_MODEL_NAME names another checkpoint.
• Routing (SRV-189): the Qwen stage returns every class's probability (the first token's top_logprobs); a code class below 0.7 for a request that names nothing of code and names a piece of prose is not taken, the best other class is (guard in the stage row). The UT2 essay went to code_edit at 0.57 (chat 0.29 behind it); on the labelled sets plus 32 prose prompts the guard changed two essays and nothing else.

Restarts

• The status responder answers GET /v1/models (and /v1/models/{id}) with 200 and the engine's last model list, kept in the status file by set up (OPS-67): Open WebUI dropped TandemLLM from its list on the 503 and a chat said only "Model not found". Chat requests keep the 503 with the engine's own sentence. The reference box's Pi model-list merger asks the responder when the engine lists nothing.

The dashboard and the ledger

• TandemLLM's own calls to itself (the router's question to the engine) are client kind internal (VIS-60): left out of Usage, the summary, the counters, the filters and Recent requests unless "show internal" is on, each naming the request it served. Ledger schema 6: parent_request; the migration relabels the router's earlier rows once (meta.relabel_internal). Contract: internal in the client kinds, parent_request on a request row, filters.internal.
• The Live panel's cached tokens are never above the client's prompt; TandemLLM's notes show as their own figure, +N hidden (VIS-58, hidden_tokens in the live rows).

0.3.42-test, 2026-10-06

0.3.41-test (a0a943a) with the dashboard's password prompts removed (VIS-56, QA VIS-57). A user report: the memories should be visible with no password prompt anywhere in the UI; when a password applies is decided later. The engine, the API and the text are unchanged.

Access

• The admin password is the admin token, QSE_ADMIN_TOKEN (TANDEM_ADMIN_TOKEN for the standalone proxy), and it is the one switch. With none set, every admin route answers anyone who reaches the port: the conductor's writes, every memory route (notes, transcripts, the disk tier), the decision models, /v1/config writes, both cache routes and the full /health. Until now they answered 404 with no token (fail closed), and with a token the dashboard asked for it. /metrics keeps the metrics token when QSE_METRICS_TOKEN is set, for Prometheus. With a password set, the rule is the one before, unchanged (server/auth.py Auth.password_set; docs/server.md "The admin rule").
• GET /v1/dashboard/session says password_set; with no password it answers login: false whatever --dashboard-login says (contract session 1.1, additive). The startup line says "admin password: not set ... keep this server on a trusted network". A conductor version written from the network with no password has the author open <address>.
• ops/make-secrets.sh writes only the metrics token unless --admin asks for the password, so making a Prometheus token no longer locks the dashboard.

The dashboard

• No password prompt anywhere: the sign-in screen, the sign-out button, the token dialog ("Admin token needed", "Unlock editing"), the Memory view's sign-in state and the token kept in the browser are gone, and no request carries a token. A token an older build stored is deleted on load. Conductor > Memory shows its transcripts, notes and disk tier at once, and a note is edited and saved straight away. With a password set on the server, a 401 shows a text banner and the views show the error; the sign-in for that comes later.
• The box: QSE_ADMIN_TOKEN is out of secrets.env (the old file kept as a backup); the metrics token stays for the Pi's scrape.

0.3.41-test, 2026-10-06

0.3.40-test (0c48e92) with the eleven fixes from a user-level self-test the same morning (UT1: 23 scenarios as an ordinary user would run them, raw API, opencode, the dashboard at 1440 and 390 px), and one test-helper race. A request without these shapes is served as before, and the raw path is untouched; the text of the shipped testing packages changed (below).

Requests and usage

• A malformed request is a 400 that names its field: empty or missing messages, an unknown role, a system prompt alone or only tool results (no user message), a system message after the conversation began, max_tokens or max_completion_tokens below 1, an empty completion prompt, a body that is not a JSON object, and a prompt that renders to no token. They used to be 500 internal_error with the chat template's exception text, which OpenAI clients retry, and max_tokens: -5 was served as a one-token answer. OpenAI's developer role is the system prompt, and a system plus a developer message become one system prompt (SRV-167; tests/test_compat.py).
• usage.prompt_tokens is what the client sent. The router's notes (this turn's and the session's replayed ones) and the closed think block of a turn it turned thinking off were counted: a "hi" reported 238 prompt tokens under the chat profile, 11 raw. They are usage.prompt_tokens_details.hidden_tokens now (absent when there are none), cached_tokens stays within the client's count, and the Playground shows a "hidden" figure (SRV-165; tests/test_conductor_e2e.py checks the count against a server without the conductor, turn after turn).
• A JSON answer (response_format json_object or json_schema) whose client names no reasoning_format gets the thinking in reasoning_content, so json.loads(message.content) works; it also gets the answer reserve, so the thinking cannot spend max_tokens and cut the JSON, and it carries no state lines. A client that asks for tags keeps them (SRV-169).

The conductor

• The bare compaction shape (no tools, no system text, one user message of 2,000 characters or more) is a compaction only when the client is opencode. A pasted document with "summarize" from any other client had that shape and was journaled as a compaction, thinking off, with the line "the conversation was compacted" (SRV-175).
• One user (a design decision, 2026-10-06): every client is the engine's one user, owner, so a curl "remember that ..." is a note in that user's store, as it should be. A request keeps out with X-Tandem-Memory: off (no card, no read, no write, journaled why: header) or read (the reads, and a block the model writes anyway is hidden and dropped). Live checks and tests send off (SRV-173; docs/memory.md).
• The waiting line said "waiting while the shadow model reads the conversation, 1st in line, about 10 s left" every time: the shadow job yields at its next chunk, so it counted as 0 s, and the rounding made every estimate "10 s". The ledger's waits behind it were 0.1 to 3 s. The line is in plain words now ("TandemLLM finishes its background work first"), there is none for a wait under 2 s, the estimate behind such work is the median of the waits measured behind it, and below 10 s it reads in seconds. Each request's time behind TandemLLM's own work is the ledger's wait_own_ms (schema version 5, added in place) (SRV-177).

The testing packages

• They named the provider by a literal LAN address (https://ai-api.example.lan) where the token belongs (the box's saved packages, versions 3 to 7). On the Spark, whose trust store lacks the LAN's private root CA, every Laya call failed TLS, and the dupfinder session spent 94 s of 222 s (42 %) debugging certificates. The texts use {tandem_url} only, and a package that calls /v1/laya, /v1/library or /v1/memory at a literal address is refused at save. QWEN38_TANDEM_URL sets the address the notes give instead of the request's own. The Laya helper exits 3 with one line when the call fails ("the call failed on this machine, your code is not the cause"), and laya-testing tells the model to go on without the verdict (SRV-163).
• Scratch files (the logs, the Laya helper, the playtest kit and its log, pid files) go to {tandem_scratch}, .tandem in the project by default, which the model adds to .git/info/exclude when it creates it and deletes as its last step before "done"; QWEN38_TANDEM_SCRATCH names another place (/tmp/tandem-{token} for a client without a directory sandbox: opencode asks before any path outside the project, and its run mode rejects it). In all three opencode sessions of the self-test the user's project ended with .tandem/ in it. A state hook (tandem_state, /__state, --dump-state) is scaffolding the model wraps in tandem-test-hook comments and removes before "done" unless the user asked for one, and the model never downloads a runtime or an executable (the todo session fetched bun from GitHub) (SRV-171).
• tools/packages_migrate.py applies these edits to the shipped texts, a box's saved packages document (a new version, the old ones kept) and its skill copies; a second run changes nothing (tests/test_package_texts.py).

Operations and the dashboard

• While the engine is away, the service says so. ops/start.sh, stop.sh and hold.sh write logs/engine-status.json (the state, since when, a hold's reason and budget, the last five start times), and ops/engine_status.py's responder (QSE_STATUS_PORT, 127.0.0.1) answers the reverse proxy's fallback: a page for /dashboard/ that says what is going on and opens the dashboard when /health answers, and an OpenAI-shaped 503 with Retry-After for the API. The proxy's old fallback said "vLLM cold start takes ~5 min" for a 22-minute hold. An open dashboard shows the same facts in its offline banner (OPS-65; docs/operations.md has the nginx snippet).
• The admin-token dialog closes on Escape and on a route change, gives the focus back, and is one dialog for calls that need it at once; a read says "Admin token needed" for its view instead of "Unlock editing", and after Cancel the Memory view shows a sign-in state. It used to stay over the next page and block the sidebar (VIS-52).
• /v1/dashboard/system says proxy.mode: off on an engine without --backends, and the Backends view reads it first: it no longer asks /v1/config (a 404 in the console on every visit and poll) (VIS-54).

Tests

• New: tests/test_package_texts.py, tests/test_engine_status.py, dashboard/tests/away.test.ts; added to: test_compat.py, test_usage.py, test_conductor_e2e.py, test_conductor_router.py, test_compaction_turns.py, test_memory_router.py, test_queue_wait.py, test_fake_engine.py, test_laya_drive_kit.py (the kit runs from a scratch copy), the Playwright specs memory and backends.
• tests/proxy_util.py: rows() waits until every completion request the test sent was accounted. The relay accounts after the client has read the answer, so under the box suite's parallel load test_proxy_relay.py sometimes read the ledger first.

0.3.40-test, 2026-10-06

0.3.39-test (ab89d54) with parallel requests merged (0.3.38-test, below; --parallel N, off by default) and four fixes from an Open WebUI agent chat on 2026-10-06 at 06:11 (five requests, one session): the client turned thinking off and the router said it was on, a stopped looping call sent back five times byte for byte, the loop guard's latency, and the client's no-repeat mask on code. With the flag off and none of these shapes the request and the stream are the release's.

Thinking off is the client's, and says so

• Open WebUI's agent model sent reasoning_effort: "none". The template closed the thinking block (the ceiling rule: none is off), but the router's row read thinking: on at the class's xhigh, and its note told the model "Reasoning effort is set to xhigh. Please think carefully ..." in a turn that could not think.
• An explicit off in any form is now the row's thinking: off_client, no budget, and extra.client_thinking ("client turned thinking off", the field, the level that did not run); the dashboard reads "thinking off (client)". The effort sentence is left out of such a turn (effort_note_skipped:client_off); a package's and the class's rules stay. The 10-03 ruling stands: the client's word is a ceiling and none is off, under every profile.

The retry storm after a stopped call

• The A1 guard stopped the write_file call at 6,400 tokens (a 405-character unit, 7.3 copies). Open WebUI answered the never-closed call with a tool error and sent the conversation again with its 6,400 tokens of arguments in the history; the model copied them, 5 times (about 55 s each, the prompt 17,692 to 46,419 tokens). The note at the prompt's tail changed nothing.
• server/argloop.py collapse_history: an assistant call whose arguments do not parse (or a value of opencode's invalid stand-in) whose tail is periodic keeps its first 1,500 characters and one marker; a call that parses is never touched. The incident's arguments: 15,444 characters, 1,756 kept. Done before the router, the monitor or the template read the messages; --argloop off|log leaves the history alone.
• The retry streak is read from the markers after the last user message. The retry's note names the tool and the piece and says when the same loop came again; the router thinks one level up (at least medium) on it unless the client turned thinking off (extra.argloop_retry).
• After two identical stops in a row (or three of one tool) the next request is not generated: A1's GIVE_UP (tool_arg_loop_retries(n)), one sentence, finish stop, no call, so the client's tool loop ends. Served like a cached answer, nothing remembered.
• tests/test_argloop.py 8 and 9: Open WebUI's loop on the fake engine, five attempts become two generations and the sentence; the real template's prompt carries the marker, not the loop.

The loop guard at three copies of a long unit

• A unit of 256 characters or more fires at 3 copies over 384 tokens; shorter units keep 6 copies over 1,024 tokens. The incident's call, replayed (tools/argloop_fp.py --replay): stopped at token 5,760, 448 tokens after the loop began, against 6,336 and 1,024. Synthetic loops of 405 to 4,000 characters stop at about half the tokens (714 against 1,418 at 405).
• False positives (tools/argloop_fp.py, the served tokenizer): opencode's 4,780 legit writes and edits, 0 fire before and after; 9 synthetic legit shapes that repeat on purpose (sprites, identical frames, tile maps, tables), 1 fires before and after (600 zeros written out, 38 identical lines). Three identical frames in a row do not fire; a fourth would. The numbers are in docs/server.md.
• tools/argloop_fp.py: the tiers' worst legit stretches, --synthetic, --replay.

The client's no-repeat mask on code

• The same requests carried no_repeat_ngram_size: 4: 4 nats off every token that would repeat a 4-gram of the conversation, which in code (indentation, pixel rows, ", ") is the text itself. Under the conductor a client's no_repeat_ngram_size is now dropped on the code classes' turns and on every turn that offers tools (extra.penalty_dropped, a [penalty] log line); the server's default stays and raw requests keep it. No A/B was run, so nothing here says the mask caused that chat's degeneration.

Parallel requests in 0.4.0

• parallel-a (66a76a9) merged; conflicts only in VERSION and this file. docs/parallel-requests.md (new): switching it on, the flags, the memory, the numbers, the text.

Tests

• tests/test_argloop.py 7 to 10 (new, 11), tests/test_client_budgets.py (the client-off row, no effort sentence, the penalty rule), the dashboard's didWhat.
• The box's CPU suite (205 files) on the fixes: 205 of 205. On the merge alone 204 of 205: test_proxy_relay.py failed under the suite's four-way load and passed alone. The dashboard's vitest: 403 of 403.

Measured on the box, and live (single runs, needs verification)

• Identity gate against 0.3.39-test (ab89d54), flags off: PASS. One request at a time (the atlas row, store off, --no-learn-cost): 0.3.39-test 49.96 mean tok/s (93.4 ms a block, p90 62.04), this release with --parallel off 50.39 (93.5, p90 62.09).
• Parallel requests after the merge (--parallel 4 --side-max-len 8192, the four bench prompts, one pass): one at a time 48.35 (median), two at once 65.83 in total (1.36x), four 67.00 (1.39x); every two- and four-request window showed that many slots decoding at once (59 and 80 two-way rounds, 45 four-way), no timeouts. PAR1's numbers were 48.8, 67.4 and 69.2.
• Live on :8000 after the deploy: smoke 27/27 raw, on and coding; ledger rows kept. A tool-call request with reasoning_effort: none: the row reads thinking: off_client (extra.client_thinking: reasoning_effort, level xhigh), the effort part was not injected, 0 reasoning tokens, the call made. The same request without an effort word: xhigh, thinking on, the effort part injected, 98 reasoning tokens before the call.
• tools/argloop_live.py --owui (the incident's shape: Open WebUI's agent loop, a 412-character frame the prompt asks to write out 40 times by hand): with reasoning_effort: none the first call stopped at 1,024 tokens (4 copies, the long rule at its first check), the retry's prompt grew by 198 tokens instead of the call's length (the history cut, 1,837 characters to a 50-character head and the marker), the model looped again on the explicit instruction, and the third request got the give-up sentence in 0.0 s with no call: three attempts, 23.5 s, where the incident ran five of about 55 s. With thinking allowed: the same three attempts (520 reasoning tokens each; the class was already xhigh, so the retry could not think higher).
• A tool request with no_repeat_ngram_size: 4: [penalty] ... dropped (class chat, tools yes), no penalty in force.

0.3.39-test, 2026-10-05

0.3.37-test (2186a22) with memory recall in a new session fixed, and three release leftovers. The request and the stream are the release's with memory off (identity gate against 2186a22); with memory on, the matching notes' note reads differently.

Recall in a new session

• The report (CLIP1's demo, live 0.3.37-test, coding profile, memory on, 21:35): a new session asked "Which node version does the build of this project need?", the ratified note "build requires node 20" showed uses 1 and body_notes 0, and the model searched the files and answered from them.
• The body was attached. The journal row of that request has bodies: [6ac3fbaed97d] (BM25 11.8), and the prompt carried the card (227 tokens), the index (180) and the body (113) at its tail. Two things were wrong:
• body_notes counted nothing on a first turn: mark_injected added the index's and the bodies' ids to index_notes whenever the index went in. Each is now counted as itself.
• The note asked the model not to use it. Every hidden note is framed by the router's wrapper, "Follow it and do not mention it", and the bodies' head only said they match. The model never named the note and went to the files. The bodies now have their own frame: the notes are the user's, the model may tell the user what one says; the head asks it to answer from a note that answers the message, to say it comes from its notes (a ratified note is confirmed by the user), and to tell the user both when the files disagree. The other notes keep the wrapper. Still a tail note, within the same caps.
• The model can no longer archive a ratified note. In the recall check before this deploy (live 0.3.37-test, 22:33), a fresh session read the note, found no Node build in the demo's Python project, and archived the user's ratified note ("› forgot: build requires node 20"). Ratify, pin and delete are the user's: the model's archive of a ratified note now makes it provisional again, with its reason in the note's changelog, and the client gets no forgot line. Its own provisional notes it still archives. tests/test_memory_notes.py test_model_archive_of_a_ratified_note (new).
• tests/test_memory_router.py test_fresh_session_recall (new): CLIP1's case on the fake engine (the ratified project note, opencode's title request, the question in a new session): the body goes in on the first turn after the index, in its own frame, and the counts read one index note and one body.

Release leftovers

• --no-learn-cost's help no longer says a block's choice stops depending on timings: it takes out what earlier requests taught the router, and timings inside the request still size its blocks (docs/exactness.md). --relax-tau and --relax-rank say what 1 means instead of "lossless": only the target's own token is accepted, and greedy output agrees with plain greedy decoding up to a near tie.
• tools/lane_check.py compares tokens, not texts: B and A2 must be A1's tokens up to the first fork, and the fork a near tie (each run took the other's runner-up, the top-2 gap within --tie-nats, 0.125 nats, in both rows). It prints the fork and both gaps. Before, a byte compare of the texts failed at a near tie with no bug behind it. tests/test_lane_check.py (new, 5).

0.3.38-test, 2026-10-05

0.3.37-test (2186a22) with parallel requests, phase A, behind --parallel N (QSE_PARALLEL), off by default: with the flag off (the service's setting) the engine is the release's (identity gate against 2186a22: PASS) and so are the 13 plain requests of tools/api_text_check.py with the routers' learned costs frozen (--no-learn-cost, both passes byte-identical, also with --parallel 2 and one request at a time).

Parallel requests (docs/server.md, "Parallel requests")

• --parallel N builds N - 1 side slots beside the engine (server/slots.py): each a full sequence (KV, recurrent state, verify graphs, drafter stack) over the one copy of the weights, --side-max-len tokens long (65,536); slot 0 keeps the 262,144 window and every cache. A request that finds slot 0 busy takes a free side slot when it fits, has no images and does not continue slot 0's resident conversation.
• The GPU turn: every slot's GPU work runs in its turns, offered back between blocks and prefill chunks (virtual time); whoever holds the engine lock holds the turn as slot 0.
• Lockstep rounds: two or more decoding slots verify their trees in one pass (engine/model.py forward_tree_multi, every projection once over all the slots' rows), each tree cut to its share so the pass stays at or under 32 rows (16 a slot at two, 10 at three, 8 at four). A chain-shaped block verifies alone; a slot alone keeps its graphed verify.
• Admission by bytes: a request's worst case (its prefill's workspace and one state-store entry) is reserved; with others in flight it waits while MemAvailable less the caches' growth and every reservation would fall under --parallel-floor-gb (15). A side slot is built only under the same floor: 104 KiB a token of window plus 7.5 GiB (measured 6.9 to 7.5 GiB for an 8,192-token slot, most of it its 121 verify graphs' pools).
• /health, the System page and the Parallel view carry the slots, the turn, the rounds by size and the admission (only with side slots, so a single-sequence server's documents are unchanged); the Parallel view shows each engine lane's slot and the engine's slots.
• The packed draft (--parallel-draft on): the decoding slots' draft calls in one pass over the drafter, from CUDA graphs per set of drafters (engine/drafters/draft_graph.py packed_draft, from branch par).
• --parallel-cut joint (off by default): the round cuts every slot's whole tree to the pass's rows, best nodes first across the slots, instead of fixed shares. In one same-hour pair it left two requests where they were (64.2 against 64.4 tok/s) and took four from 69.2 to 76.5, with more texts moved at near ties (two requests: 0 of 4 identical to alone, gaps up to 0.116 nats), so the fixed shares stay the default.
• Tools: tools/parallel_bench.py (c1/c2/c4 totals, each request against its alone text, the concurrency proof and the round's parts from /health, --prompt-set wide for 30 prompts), tools/parallel_verify_check.py (the joint pass against each slot's own verify on the real model), tools/packed_draft_check.py.

Measured on the box (single runs, needs verification)

Single runs on the box, store off unless noted, thinking off, greedy; "needs verification" until repeated.

• One request at a time (the atlas row, --no-learn-cost, A/B/A/B in the same hours): 0.3.37-test 50.42, 49.12, 50.35 mean tok/s (93.4 to 93.5 ms a block); this release with --parallel 2 50.12, 50.82, 51.14 (93.6 to 93.8 ms a block). In one later hour: 0.3.37-test 49.29 and 49.75 (95.1 and 94.4 ms a block), this release with the flag off 49.54 (94.7), with --parallel 2 50.59 (94.6). With the clean store: 49.93 against 49.81. The 13 plain requests: byte-identical to 0.3.37-test.
• Four long-answer prompts, 384 tokens each, --parallel 4 --side-max-len 8192, packed draft on: each alone 47.8, 68.7, 36.3 and 49.8 tok/s (median 48.8); two at once 76.5 and 58.2 tok/s in total for the two pairs (38 a request); all four at once 69.2 in total (19 a request). Every two- and four-request window showed that many slots decoding and joint passes of that many blocks (59 to 80 two-way passes a pair, 45 four-way passes); none queued.
• A joint pass of two blocks takes 94 to 98 ms on the GPU, of four 114 to 119 ms; a slot's round at two requests 125 to 145 ms, at four 226 ms. The packed draft took 9 to 13 ms off the two-request round.
• Beside other requests a request's text equals its alone text up to a near tie: on 30 prompts (prose, code and chat, 256 tokens) 17 of 30 identical at two requests and 15 of 30 at four, every first difference at a top-2 logprob gap of 0.0019 to 0.118 nats. The packed draft proposes bit for bit what each slot's own draft does. The joint pass's logits equal each slot's own (eager) verify bit for bit (40 of 40 rounds, tools/parallel_verify_check.py).
• A side slot of 8,192 tokens takes 6.9 to 7.5 GiB beyond its window, most of it its verify graphs' pools.

Tests

• tests/test_slots.py (new, 15), tests/test_parallel_verify.py (new, 6: the joint pass equals each engine's own verify bit for bit on the CPU; two and three requests in lockstep write the tokens each writes alone), tests/test_parallel_app.py (new, 4: the fake engine over HTTP, three streams at once, a fourth waits, the admission holds a request, flag off one at a time), tests/test_packed_draft.py (new, 3); the dashboard's vitest and a Playwright spec for the slots.

0.3.37-test, 2026-10-05

0.3.36-test (5b04bbc) with a crash fixed: :8000 died after three hours of metrics scrapes ("Fatal Python error: bool_dealloc", 2026-10-05 13:12, the first time). The request and the stream are the release's (identity gate against 5b04bbc).

The crash

• torch 2.13.0's torch._C._cuda_memorySnapshot gives back one reference too many on every item of the tuple it is handed. Each MemPool.snapshot() took one reference off True and one off each of the pool's id ints, which are shared singletons on Python 3.11. Placement.pool_bytes called it for the narrator's and the shadow reader's pools on every Budget.memory(), and since 0.3.34-test's role_exclude every one of the 30 /metrics collectors reaches that through cache_stats(): 30 to 60 references a scrape. After 732 scrapes True reached 0 and the process died in whatever line let go of the next reference (a disk-cache gauge's lambda). Measured on the box's CPU: 1000 calls take 1000 references off True and off each id int; the call handed None takes none.
• The pools are now read from that call (every segment), filtered by segment_pool_id; one snapshot serves the calls of 0.25 s (one scrape's 30).
• server/refwatch.py: every 30 s the reference counts of True, False, None, 0 and 1; a count under half its high is logged with its fall rate and the time to 0. /health's memory part carries the counts (python_refs). Python 3.11 only: from 3.12 these objects are immortal.
• tests/test_pool_snapshot_refcount.py (new, 6): the upstream leak as measured; pool_bytes 2000 times keeps every reference (before: -2000 on True, on 0 and on the id int); the pool's segments from the full snapshot, never a tuple; the watch; the /health counts.

0.3.36-test, 2026-10-05

0.3.35-test (c718bbd) with the image path's packages in the lock and its backend pinned, the writer's switches on the Backends screen, two steadier checks, and the proxy's writer checked against real engines. The request and the stream are the release's (identity gate against c718bbd).

The image path's packages (a user report)

• A user installed TandemLLM in a plain PyTorch container and had to add Pillow and torchvision by hand. server/images.py decodes with Pillow, and transformers 5.12 builds Qwen3.8's image processor on torchvision (Qwen2VLImageProcessor, torchvision.transforms.v2) when torchvision 0.27 or later is installed, else on Pillow (Qwen2VLImageProcessorPil). Neither was in install.sh's lock.
• The lock now has pillow==11.3.0 and torchvision==0.28.0+cu130 (from the CUDA 13.0 index, the build that matches torch 2.13.0+cu130), and the README's requirements name both, with the pip line for an existing PyTorch container. install.sh's check after the install imports both and prints the image processor class it resolved.
• The backend is pinned: --image-backend torchvision (the default; QSE_IMAGE_BACKEND). The two backends are not the same numbers: after a resize a patch value differs by up to two 8-bit levels (measured on the box: 0.4 % of the bf16 values of a 1000x333 image, none for an image that is not resized), so the digest the caches key on and the tower's input depended on what was installed. Every served measurement ran torchvision. A backend that does not resolve stops the start with the package to install, never a silent fallback; --image-backend pil is the explicit choice without torchvision, and --vision off serves text only.
• No engine extra in pyproject: torch and torchvision must be the matched CUDA 13.0 pair from the PyTorch index, which an extra cannot say, and an extra resolving torchvision from PyPI could replace a container's torch. install.sh's lock stays the one list.
• tests/test_engine_deps.py (new, 8): the lock has both packages and a torchvision build that matches torch; the README and the install check name them; every import of server/images.py is the standard library, the repo or the lock; a fresh interpreter that decodes a PNG and runs Qwen3.8's processor on the pinned backend loads no package outside the lock that the image packages require, and the processor's own module imports torchvision unconditionally; load_processor refuses transformers' silent fallback. Taking pillow or torchvision out of the lock fails three of them.

The background writer on the Backends screen

• A backend's form has the writer's four switches: mode (auto, off, shadow, on), the spend cap a day with a no-cap box, runs a day, and the prompt cap. Only the fields that changed are sent (a default stays unset); a field a TANDEM_BACKEND_<ID>_WRITER* variable sets is locked and names it. The mode's hint says what auto resolves to for this kind and, in shadow, where the would-be blocks go. Phone width: one column.
• A backend's page shows the day's runs and spend against the caps, each model's resolved mode and why (auto, a paid cloud model, the backend's switch, lowered by the settings), and in shadow the log's place. Its card says writer <mode>.
• The Backends API adds writer.settings_mode and writer.shadow_log.

Steadier checks (NP1's issues 4 and 6)

• tools/proxy_gate.py: the decode check is relative. The median request within 2 % of the engine's rate, each request within 5 % on its best of 3 tries; the token counts, the status and the TTFT order hold on every try. The proxy times frames as they arrive, so over a LAN one late last frame moved a 96-token request by a few percent (1 and 4 of 20 failed a per-request 2 % from the Mac; 20 of 20 passed over loopback). The report says whether the engine was on loopback. tests/test_proxy_gate.py (new, 3): a fake engine whose last frame arrives late fails the old gate and passes the new one on its retry; a request late on every try still fails.
• e2e/performance.spec.ts: the two dot specs pick an engine-measured dot and select it with the keyboard. Every seventh mock row is relayed through the proxy and draws no timing bar, and which dot came first depended on the rows earlier specs had added, so the spec failed alone and passed in a run. The scatter's dots carry data-measured.

The proxy's writer against real engines

• tools/proxy_writer_live.py (new): the standalone proxy in front of a real vLLM, SGLang or cloud API with the writer on: a session with a fact written at its end, shadow (nothing stored, the log written), busy (a client request in flight: the look starts nothing), yield (a client request while the writer streams). On a cloud API the exact cost of every request from the ledger.
• tools/memory_replay.py (new): a real session's transcript replayed into an engine with memory on, so the writer writes from a real session (OPS-55).
• Measured on the box on 2026-10-05 (Qwen3.5-0.8B, vLLM v0.30.0 and SGLang v0.5.21 with the qwen3_coder tool parser, n = 1 each): busy held on both (no writer request while a client request is in flight); on SGLang a client request 1.1 s into the writer's stream closed it and the session stayed due. Without a prefill the 0.8B model wrote a block the schema refused (vLLM) or 400 tokens of text (SGLang); with the prefill Facts worth keeping (blocks, or NONE): it wrote 2 notes on vLLM (2,720 of 3,153 prompt tokens from vLLM's prefix cache) and 1 on SGLang. Without a tool parser vLLM refuses a request with tools (400), and SGLang does not hold the model to tool_choice: "none" (the writer's invented calls were cut to NONE). DeepSeek (deepseek-flash, writer on, one run): 1 note; the whole test spent 1,273 micro-USD at the list price (7 requests), the writer's request 287.

0.3.35-test, 2026-10-05

0.3.34-test (5b6fb5d) with the proxy's background memory writer and the writer's own NONE. The request and the stream are the release's (identity gate against 5b6fb5d); what changes is the writer's hidden turn on the engine, and a request the proxy may ask of a backend in its idle time.

The background writer behind the proxy (MEM2's open item)

• For a backend behind the proxy (vLLM, SGLang, any OpenAI-compatible server, a cloud API) the session-end and after-compaction writer is one more request to the backend the session used (proxy/memwriter.py): the messages of the session's last request as they were sent (so the backend's prefix cache matches), its answer, and the writer's user turn; thinking off; the session's tools kept with tool_choice: "none". Its answer goes through the engine writer's parser, store and journal; the run says via: proxy, backend, model, cost_micros, and the request is a ledger row of the client harness:writer (counted under memory on the Backends screen).
• In the backend's idle time: one writer request at a time, only while the backend has no client request in flight; on a local backend a client request that arrives closes the writer's stream (the session stays due). A compaction request in the session keeps the turn before it as the record.
• Per backend (harness document, TANDEM_BACKEND_<ID>_WRITER, _WRITER_USD_DAY, _WRITER_RUNS_DAY, _WRITER_PROMPT_TOKENS): writer auto (on; off on a paid cloud model), off, shadow, on; a spend cap a day on a paid model (0.25 USD, checked against an estimate before each run), a run cap a day (48), and the prompt's size (32,000 estimated tokens, the oldest messages dropped first). The effective mode is the lower of the backend's and settings.memory.notes.writer.mode; a paid model without a price is refused. The day's figures come back from the ledger after a restart; the Backends API shows the mode per model and the day's runs and spend.

The writer's own NONE (L2's open item)

• settings.memory.notes.writer.ban_tool_call (on): the <tool_call> token gets a logit bias of -100 in the writer's turn, so after a compaction Qwen can no longer answer with a bash call; the cut of 0.3.34 stays as the backstop. Behind the proxy the same switch sends tool_choice: "none".
• prefill (empty): text the writer's answer starts with (the engine: tokens after the generation prompt; the proxy: an assistant start where the backend can continue one).
• Not yet measured on the real engine: tools/memory_measure.py --ban-tool-call on|off --prefill ... --stop-after compaction is the night pass's n = 1 check.

Shadow writes a log

• A writer run in shadow appends the blocks it would have written, after the secret-pattern refusal, to <user>/shadow/writer-<yyyy-mm-dd>.md (0600), which nothing reads back. The journal keeps counts only, as before.

Tests

• tests/test_proxy_memwriter.py (new, 9): the switches and the env variables, the trim and the request body (tool_choice none, thinking off, the prefill where the backend continues), an idle vLLM session written with the prefix it sent, shadow's log, a paid cloud model (auto off, no price, the spend cap, the run cap, the spend read back from the ledger), NONE and a cut bash call, the compaction record, a client request that stops the writer, the Backends API.
• tests/test_memory_writer.py: the <tool_call> ban (the token the greedy turn chose first without it is never chosen with it), the prefill.
• tests/fake_backend.py: the writer's request (FAKE_WRITER_NONE, _BASH, _SLOW).

0.3.34-test, 2026-10-05

0.3.33-test (78a09bc) with the loose ends of the night pass and of the memory release. The request and the stream are the release's (identity gate against 78a09bc); what changes is when the shadow reader loads, how the engine's memory ceiling counts the roles' pools, and what the memory writer says.

The night-fix2 branch (the engine server's listen backlog of 128 and no Nagle, the proxy's backlog, the fake backend, SGLang's window rates) and NP1's issues 1, 2, 5 and 7 (the waiting line names TandemLLM's own work, a malformed tools is a 400, "Backend" whole at 390 px) shipped in 0.3.32-test already; on :8000 the listen queue reads 128.

The shadow reader's first load (NP1's open issue 3)

• After a restart the shadow's model loaded with the first turn of a profile with shadow.mode: on. That turn waited 11.2 s behind the load, its class question gave up after 8 s, and the next memory gate found the engine over its ceiling and shed the anchors, the end, the stash and the park (6.3 GB on 2026-10-05 06:18, the same at 21:41 and 23:34 the night before, for a 1.47 GiB model).
• The first load now runs in an idle window soon after the start (app.shadow_preload_once, a look every 20 s): only while no request runs or waits, no other model loads and some profile has the shadow on, once. After a release, the next turn loads it as before. A slot the narrator shares stays its keeper's.
• The roles' own pools are left out of the engine's figures up to the claim the residency planner admitted them with (server/memlimits.role_exclude, Budget takes an allocated and a reserved figure). Before, the shadow's own slot was not left out at all and the narrator's only by its model's tensors. Over its claim a pool's excess counts as the engine's, and its free part is what the stale-reserve rule leaves out (role_pool_free), so the gate's roles level still has something to give back.
• The slot keeps the planner's claim, its placement is in place from the load's first copy (a request between two copies sees the pool as the role's), and a failed load drops its pool. The load's log line said "MemAvailable --2.87 GiB" when MemAvailable rose over the load; it says "+2.87" now.

The memory writer (MEM2's open items)

• A tool call that is not a tandem_memory call ends the writer's turn at its function's name: after a compaction Qwen answered an empty or a bash call, which writes nothing however long it runs. The run says what NONE says: none: true, answer (none, empty, tool_call, text, blocks), cut (the function's name, or empty), finish: tool_call; the log reads wrote: NONE (tool_call: bash, cut). The Notes tab shows such a run as "nothing worth keeping". Whether a prefilled answer makes Qwen write NONE itself is still to measure.
• A resident conversation the memory never saw (no session id, raw, a profile with memory off) is the look no_session, printed once as writer look: no session (...); it printed nothing with the record's numbers.

Tests

• tests/test_roles_memory.py (new): the pools up to their claims, the gate after the shadow's load (nothing shed; the old figure shed four levels), the slot during its load, the first load ahead (idle only, once, a profile on, not while another model loads, not the narrator's slot, a refusal said once).
• tests/test_memory_writer.py: the foreign call cut and the run's NONE for a bash call, an empty call, an empty answer, NONE and prose; no session said once.
• tests/test_proxy_relay.py: the user's row is found by its status, not its position (the box suite under a 4-way load wrote the 401's row after the 200's).

0.3.33-test, 2026-10-05

0.3.32-test (ed7aaec) with the memory notes (branch tandem-mem: P-M1, P-M2, P-M3 and the writer's fixes). Every profile's memory switch is off by default and the writer's mode is off: with both off the request and the stream are the release's (identity gate against ed7aaec). Docs: docs/memory.md.

The store, the hidden block, the reads (P-M1)

• The model keeps notes as files under ~/.qwen38-spark-engine/memory/notes/<user>/: one fact a file, three scopes (user, project, session), the eight kinds, revisions, an archive, an INDEX.md the nightly job writes (server/conductor/notes.py). Caps are refusals; every note passes the secret-pattern refusal; a rule about the user needs the user's message in the turn; the model writes provisional notes only; an add of the same fact is a revision.
• The write: a <tandem_memory op=add|update|archive ...> block in the answer, held back from the client like a tool call and written at once (server/memblock.py); a state line › remembered: <title>; a block that is not taken goes out as text. Its tokens are recorded as hidden spans, so the end anchor's replay keeps the next turn's prefix (a block whose edge shares a token with text is not, and that turn prefills from there).
• The reads (profile memory: on): the memory card, the index once a session and after a compaction, the matching notes on a new user message, the pinned rules; each a tail note replayed at its place. Laya's remember question attaches the memory skill (shadow by default). New settings memory.notes and profile switch memory (off, shadow, on).
• The model's scripts: GET|POST {tandem_url}/v1/memory... under the session's address only (server/memory_api.py).
• Dashboard: Memory has a Notes tab: the tree by scope, the list with uses in 7 days, a note as a file (edit as a ratified revision, ratify, reject, archive, pin a ratified rule, delete), its history as diffs, the runtime's search with scores, the requests the notes went into, the writes and refusals, the notes settings, export as a zip, the user's delete behind a typed confirmation. 390 px: the tree is a picker. Routes /v1/conductor/memory/notes..., admin token on all of them, reads included.
• server/conductor/libsearch.py: the BM25 index is TextIndex, shared by the library and the notes; the library's scores do not change.
• The fake engine writes a memory block on FAKE_MEMORY (_PROJECT, _BAD, _OPEN).

The background writer, the proxy's memory group (P-M2)

• The writer's hidden turn (server/conductor/memwriter.py, app.memory_once): Qwen, thinking off, continues the session's conversation with one user turn that asks for <tandem_memory> blocks and lists the notes that already match, at the session's end (idle_s 180 s, min_new_tokens 1,500), after a GIVE_UP and after a compaction (after_compaction: idle, the default, or turn), behind Laya's gate. A request that arrives stops it at its next chunk; the rows stay as they were. Writer runs in the Notes tab. Settings memory.notes.writer (mode off, shadow, on).
• The proxy's memory feature group and the X-Tandem-User header (the proxy user's name, accepted from loopback or QWEN38_MEMORY_PROXIES).
• The writer never wrote on the box (MEM1, 01:16): every fix below is from the measurement on a test engine (tools/memory_measure.py, 4 runs in holds 04:50 to 05:22).
• Inside a hold the box lock is held, and every idle look answered "hold" without a word. The look's outcome is now in the writer's stats (writer.last_look in /v1/conductor/memory/notes/writes) and printed when it turns to hold, gpu, nothing or moved; a test engine in a hold takes its own BOX_LOCK.
• A compaction whose prompt was short enough to be served as a guest had no record in place, and the writer returned without a word. Its mark now stays, and the next idle look writes the session (trigger: compaction).
• The record from the rows' last prompt ended in the generation prompt (<|im_start|>assistant\n<think>\n), so the writer's turn followed an open assistant turn: 48 tokens and no block. It now stops before the last message's <|im_end|>.
• Laya's gate said no to "Remember this for the whole project: every function needs a one-line docstring ..." (0.77) with the first wording. The new words, chosen on :8000's laya-english with 4 sessions that hold a fact and 4 that do not: yes to the 4, no to 3 of the others.
• With the session's tools in the record, Qwen wrote the blocks as <tool_call><function=tandem_memory> calls; a call of that name now counts as its block, and the turn says it has no tools.
• No gate after a compaction in the turn: the in-process decider asks only while no request holds the engine, and under the lock it waited out its deadline.
• Measured on the box (test engine on :8011, a synthetic 10-turn opencode session, low effort, writer on, after_compaction: idle): the writer at the session's end wrote 2 notes (a gotcha and a program_state) next to the 3 project rules the model had written inline, 7.6 s for 282 tokens (5,374-token record, prefill 0.8 s), gate 33 ms. In the run before it, with no inline block from the session, it wrote the 2 rules the user had asked to keep plus a gotcha and a program_state (388 tokens, 7.6 s). After a compaction it ran in 2.1 to 2.9 s and wrote nothing (the notes it was shown covered the facts). A new session that asks what is remembered: 1,781 prompt tokens with memory on (index and 3 notes) against 136 off, first streamed token 0.97 s against 0.39 s; the answer names both rules (off: neither). Decode, one coding question in new sessions, 3 alternating pairs: 99.7 ms a block with memory on, 99.6 off (engine side; the tokens a block follow the drafters' acceptance, which differs by answer).

Package proposals (P-M3)

• op=propose writes a proposal under memory/proposals/<name>/<n>.md with its evidence (the session's J1 low-quality and stuck rows, or none); the model never applies it. The Packages screen's Proposals tab shows the diff against the live text; Apply stores the text as a new packages version (the old version restores it); a replacement of a text that changed since is refused, an append applies to the text as it is.

Tests

tests/test_memory_notes.py (16), tests/test_memory_router.py (8), tests/test_memory_writer.py (9: the writer writes and leaves the rows, a waiting request stops it, not due, the gate and the marks, shadow, a held box lock, a compaction without its record, the prompt record's end, the tool form), dashboard tests/notes.test.ts, e2e notes.spec.ts (Proposals included). On the box: the CPU suite 195 of 195, vitest 396 of 396, the memory, notes and packages e2e on Chromium 28 of 28.

0.3.32-test, 2026-10-05

0.3.31-test (5be6f46) with two branches merged: the parallel view (branch parallel-gui) and the night's fixes (branch night-fix3, which carries night-fix2). --backends stays off on :8000. Qwen's text does not change (identity gate). Dashboard contract parallel 1.0 is new; nothing else in the contract changes.

The night's fixes (branch night-fix3)

• The engine's own server has a listen backlog of 128 and no Nagle, as the proxy and uvicorn's servers. Measured on the box with the old defaults: each answer on a kept connection waited 41 ms for the reader's delayed ACK, and 5 of 64 clients that connected in the same moment got a reset; now 0.4 ms and 64 of 64. A stream closes its connection, so this matters for JSON answers on a pooled connection (the proxy's relay with --backends on). Test: tests/test_app_loop.py.
• The waiting line names TandemLLM's own work when that holds the engine: "waiting: the conductor's question to Qwen runs first, position 1", and the same for the shadow model's reading, the summary prepared ahead and a lock no request holds (the drain, a cache clear). Before, a turn behind the router's own class question read "waiting for another request to finish". The voice says it in English and German. Test: tests/test_queue_wait.py.
• A malformed tools (not a list, an item that is not an object, a function without a name, parameters that is not an object) is a 400 with param: tools that names the item. Before, it reached the chat template and came back as an error frame with a traceback in the log. The flat tool objects the template also reads still serve. Test: tests/test_compat.py.
• The phone's bottom bar says "Backend" (it cut "Backends" to "Backen…" at 390 px). e2e: every bottom-bar label shows whole on webkit-phone.
• From night-fix2 (NP1): the standalone proxy's listen backlog of 128, the fake backend without Nagle, the engines check's prefix, rounding and timing on real vLLM and SGLang servers, and an adapter counter that a window's first reading lacks starts at 0 (SGLang's rates were None for 5 minutes after a restart).
• The parallel view, found under load on the box: an engine backend's series read 145 next to 0 at turn boundaries (the end counted the tokens since the engine's last read a second time, and the engine's board was read every 2 s, out of step with the samples); each lane now sums to its count and the read comes 0.15 s before each sample, and the fake engine at 60 tok/s reads 48 to 69 a second. A backend card kept growing a second line of dots and every lane below moved (layout shift 0.14 in 20 s at N = 32, now 0.024). The chart's legend says "waiting", not "idle", for a backend with requests in flight and none decoding.

Every request at once (branch parallel-gui)

• Performance has two pages, Engine and Parallel (#/performance?tab=parallel). Parallel shows every request in flight on every backend (our engine, vLLM, SGLang, cloud APIs) at once: the aggregate decode rate now and stacked by backend over up to five minutes, the requests by state, TTFT p50 and p90, cache hit, cost per minute, tokens out and errors over the last five minutes; one card per backend; one lane per request (backend and model, client and session, state, tokens with thinking apart, tok/s now and on average, TTFT, prefill, cache, cost so far, elapsed) that keeps its row while it runs. 390 px: the lanes become cards. Docs: docs/proxy.md "Every request at once".
• GET /v1/dashboard/parallel (server/parallel.py) on the engine and on the proxy: built at most every 0.4 s for every viewer, since returns only the new series samples. With 32 streams: about 1 ms to build (20 ms at most, beside 32 relay threads), 55 KB a poll at most.
• Live counts are measured or null. The engine's own requests: its count. Our engine behind a proxy: the relay sends X-Tandem-Relay-Id, the engine files the request under it, the proxy reads the engine's own board once a second (its frames are blocks of several tokens). Any other backend: its stream frames once two answers showed frames equal to usage.completion_tokens, text answers and tool-call answers checked apart. DeepSeek's reasoning and content frames are one token each (8, 14 and 7 reasoning frames for 8, 14 and 7 reasoning tokens); its tool-call arguments are several tokens a frame (56 of 618 turns with calls matched), so a DeepSeek lane writing a call says "not reported".
• tools/parallel_load.py: N agent-like sessions (system prompt, two tools, up to three streamed turns, tool results fed back) over the models named in --mix, with a budget cap and --board (the page's series beside the frames the clients received).
• The standalone proxy (python -m proxy, tandem-dash serve) now starts its live sampler; before, its live stream never ticked.
• Measured on the box through a test proxy (DeepSeek, our engine on :8000, the fake backend, the fake engine as a second TandemLLM backend), 0 errors in all three: N = 8, 526 tok/s aggregate (294 turns); N = 16, 896 tok/s (552 turns); N = 32, 1,495 tok/s (1,125 turns; deepseek-flash 911 tok/s over 10 sessions, TTFT p50 652 ms, cache hit 69.5 %), again 1,434 tok/s (960 turns). The page's per-second series against the frames the clients received at N = 32: 1,762 against 1,861 a second; the difference is what has no live count (DeepSeek's call arguments, the engine on :8000 before this release).

0.3.31-test, 2026-10-04

0.3.30-test (0d4c7c9) with three branches merged: the loose ends (branch tandem-d9), the plugins inside vLLM and SGLang and the launcher (proxy P6, branch proxy-p6) and the night pass's fixes (branch night-fix). Usage ledger schema version 4 (four columns added in place at start). --backends stays off on :8000. Qwen's text does not change (identity gate); the coding profile gives the voice and the shadow 2.5 % of Qwen's time instead of 3 %. The other two sections below are their branches' entries as written.

The night pass's fixes (branch night-fix)

• The voice and the shadow's in-turn reading share one budget of Qwen's time (narrator.Share). With both on at 3 %, Qwen's decode lost 3.34 % (63.33 tok/s off, 61.22 on, the same text, 504 blocks each, --no-learn-cost); at 2.5 % it lost 2.79 %. The shipped coding profile sets ui.live.share: 0.025; the box's coding profile has it since v19 (no restart). A profile with the voice alone keeps 3 % (it measured -1.90 %).
• tandem-dash serve from the wheel died at start since P4: the standalone proxy runs the conductor, whose schema reads packages/seed.json, and the wheel did not carry packages/. It now ships as proxy/packages, and the conductor finds the texts there when no checkout holds them (server/conductor/packages.py shipped_root). tools/package_check.sh passes again: a 988 KB wheel, 61 of 61 files of packages/ in it.
• tools/argloop_live.py allows the one call a live-streamed call keeps (0.3.28-test) and checks that the looping block is not sent again as content.
• Dashboard e2e: the "Last 20" table test clears the mock and pauses its engine first; after dev.spec.ts in one run it saw 14 rows instead of 10.

Loose ends (branch tandem-d9)

• A live tool call cut by the token limit is sent once. A block that streamed its arguments live and was cut by the token limit (or the deadline, or an end token inside a value) went out a second time as content in ToolCallBuffer.finish, so the client's next prompt carried it twice (2026-10-04 16:18: 65,536 tokens of write arguments, a 157,806-token next prompt). Now the client keeps the calls the stream closed (they count as the answer's calls, under the ids they streamed with) and the open call as it was sent, whose arguments never close; nothing of the block is content. A block cut before its function's name streamed, and a request with stop strings (calls are not streamed live there), keep the old behaviour: the text it is. Tests: tests/test_toolcall.py (2 new, 1 changed), tests/test_argloop.py 5.
• A second session's turn is no longer refused after minutes (an open item of 0.3.27-test). A streamed chat request that finds the engine busy waits as long as one request may run (--request-timeout) instead of --queue-timeout: after 10 s its stream opens and an SSE comment : waiting, position N, about M s, T s so far goes out every 10 s, and with the profile's progress lines on the waiting line carries the estimate ("waiting: another request is running, position 1, about 40 s", new line kind queue_eta, switched with queue). The estimate (server/queue_wait.py) is the median remaining hold time of earlier requests that ran as long as the holder has, plus each waiter's typical time; with too few recorded holds the line gives the position only. A request without a stream keeps the 429 after --queue-timeout. The live view's queue.timeout_s is the request's own limit. Two full turns are not interleaved (docs/server.md says why). Tests: tests/test_queue_wait.py (4 on the estimate, 3 through the handler); tests/test_state_channel.py and tests/test_state_inline.py close the early-opened stream at the new limit; tests/test_ui_states.py starts each test with no hold times.
• For a public repository: no host, path or personal name from the development setup outside the copyright and citation lines, and no agent names. The stall watchdog's push URL is no longer in the shipped policy: notify.url is empty in ops/supervisor.yaml (and its JSON copy, the contract's seeds) and the push goes to $TANDEM_NOTIFY_URL then ("failed: no url" without either; a box whose stored monitor document names a URL keeps it). tools/tandem_eval.py --base defaults to $TANDEM_EVAL_BASE, else http://127.0.0.1:8000/v1; a suite's ~/ rewrite entry also matches the folder under any /Users/<name>/ or /home/<name>/, so suites.json lists no absolute home. The maintainer's name in comments, docs, tests, the dashboard's mock and e2e texts, ops/ and this file's history reads "the owner"; private note titles and a host alias in rollback lines are gone; tests use /home/user; the agent names and the model that wrote the skill library are out of the history, the docstrings and the comments (the workstream tags S1, J1, C1 and the like stay: they name features). Dashboard rebuilt. Tests: tests/test_session_tasks.py (the URL from the environment), tests/test_tandem_eval.py (the rewrite), tests/test_defaults.py reads the holder's name from NOTICE.
• The shadow's router question with other small models (tools/shadow_router_eval.py, docs/shadow-reader.md): on 58 journaled turns, agreement with the router 8 (0.8B), 25 (2B) and 19 (4B) of 58 with the live question; the 4B with the classes' short labels 44 of 58, which is what a constant code_edit scores (42). The shadow keeps the 0.8B; nothing changes in serve.env.

The plugins inside vLLM and SGLang and the launcher (proxy P6, branch proxy-p6)

On branch proxy-p4 (the harness in front of any backend, on proxy-p23; not released). Usage ledger schema version 4 (four columns added in place at start). Dashboard contract 1.3, additive. New tandem-dash launch. Nothing changes for the engine's own model, and a server with --backends off loads none of it.

• Two packages in plugins/ (plugins/README.md): tandemllm-vllm-plugin, a stat logger vLLM loads from the vllm.stat_logger_plugins entry point, plus an optional GET /tandem/stats route (vllm.endpoint_plugins); and tandemllm-sglang-plugin, a general plugin SGLang loads from sglang.srt.plugins in each of its processes, with two hooks (TokenizerManager.collect_metrics, SchedulerMetricsCollector.log_stats). Per request: queue, prefill and decode time, cached tokens, preemptions (SGLang: retractions), the engine's finish reason, and on SGLang the speculation verify passes and accepted drafts. Once a second: the scheduler's state. They keep 10,000 requests and 300 samples in memory, post once a second to the proxy with a shared token, and never stop the engine (an import failure, a renamed field or a failing hook is a log line and a null). No dependencies; checked against vLLM v0.30.0 and SGLang v0.5.21.
• SGLang v0.5.21 has an entry point group after all (sglang.srt.plugins, with a hook registry), so the SGLang plugin needs no launcher and no ServerArgs.stat_loggers; the tokenizer collector's per-request call carries no request id in v0.5.21, while collect_metrics has the request's state.
• The proxy: POST /v1/collect/<backend> (proxy/collect_api.py) takes the batches. The relay tags each request to a vllm backend with X-Request-Id: px-<hex> (vLLM reports chatcmpl-px-<hex>, the row's own id) and to an sglang backend with the body's rid; a client's own id is kept. The row then gets the engine's queue_ms, prompt_ms, decode_ms, decode_tps, prefill_tps and cached_tokens, and the new columns preemptions, engine_finish, corrupted and l2_join. An item that comes after its row amends it in place (Ledger.amend); one that comes first waits; either waits 10 minutes, then it is dropped and counted. A request the proxy cannot tag is matched by time and token counts when exactly one row fits (approximate, one log line). With nothing pushed for 25 s the proxy reads vLLM's /tandem/stats instead.
• The collect token: TANDEM_COLLECT_TOKEN, else a random one stored in the config database at the first need.
• tandem-dash launch vllm|sglang --model M --port P [-- engine args] (proxy/launch.py): registers the backend, hands the engine the collect URL and token, adds the flags the dashboard reads, checks that the plugin is installed in the engine's Python, then becomes the engine (the pid it records is the engine's). Without the plugin it starts the engine at L1 and says how to install it.
• Contract 1.3: engine.plugin and engine.layer: L2 on a backend's status row, l2 on a requests row, plugin_age_s filled. On /metrics: qse_backend_plugin_age_seconds, qse_backend_plugin_joins{backend,join}.
• The dashboard: "L2 plugin" or "L1 only" on each vLLM and SGLang card, with the plugin and engine versions, and a line with the joined rows, preemptions, KV evictions and vLLM's acceptance per draft position; a relayed request row whose split came from the engine says so.
• Tests: tests/test_proxy_plugins.py 12 (both plugins on stand-ins built from the tagged sources' classes and call shapes: the items, the per-second fold, failures that stay inside, the import failure vLLM's loader catches, the pull route and its token, the poster's batches, heartbeat and back-off, the two packages' shared file and license), tests/test_proxy_collect.py 12 (the route and its token, both join orders, the client's own id, SGLang's rid, the approximate join and its refusal, the waits, the ledger amend, the plugin block against the contract, L1 only after 60 s, the pull, launch with a stand-in engine); dashboard: vitest 373, e2e engines.spec.ts (chromium, webkit at 390 px). A smoke on the Mac: the plugin's poster to a standalone proxy in front of a fake vLLM, the row joined exactly.
• Not verified against running engines yet. The night pass (tools/proxy_e2e.sh --box --engines, now with tools/proxy_engines_check.sh --plugins) installs both wheels into the release images, checks the joins and the plugin's sums against each engine's /metrics within 1 %, and measures the plugin's cost on vLLM with tools/proxy_plugin_ab.py: decode tok/s without, with and without the plugin, 20 requests each, gate 1 %.

0.3.30-test, 2026-10-04

0.3.29-test (97a0ef3) with three proxy branches, each a fast-forward of the one before: the host collector and the package (P1.5 and P5, branch proxy-p15), vLLM and SGLang at L1 (P2 and P3, branch proxy-p23) and the harness in front of any backend (P4, branch proxy-p4), plus 162cbbc (columns added to config schema v1 after its first files are added in place). New server flag --host-collector on|off (on). Dashboard contract 1.3, additive. --backends stays off on :8000, so a request to the engine's own model takes the same path as in 0.3.29: Qwen's text does not change (identity gate). The sections below are the three branches' entries as they were written.

The harness in front of any backend (proxy P4, branch proxy-p4)

On branch proxy-p23 (vLLM and SGLang at L1, on proxy-p15). New backend settings harness.mode and harness.profile acted on (TANDEM_BACKEND_<ID>_HARNESS, new _HARNESS_PROFILE); new standalone flags --conductor-config (default ~/.tandemllm/conductor, off) and --conductor-defaults. A new optional words block in the monitor policy's D1 task; the engine never reads it. With every backend's harness off (the default) a relayed request is P1's, byte for byte, and the engine's own requests take the same path as before: the router's new arguments default to nothing and its stage override answers nothing for them.

• The conductor in front of other backends (proxy/harness.py, docs/proxy.md): the router's plan for each relayed chat completion (the engine's own conductor in --backends, one of its own in the standalone proxy), its note at the end of the newest user message and replayed per session at the same message index while the messages up to it hash the same (server/conductor/inject_messages.py), the effort through each kind's own switch (chat_template_kwargs.enable_thinking, reasoning_effort on OpenAI's reasoning models, DeepSeek's thinking.type), the class question asked of the backend with its constraint field (vLLM structured_outputs.choice, SGLang regex, OpenAI a JSON-schema enum, our engine its grammar, else plain text), the side requests' caps, the TandemLLM section on a compaction request, the state lines, the session tokens, the transcripts and the journal.
• The monitor over HTTP: E1 and the tool-call regenerate as second requests whose answer the client gets as the continuation of the first (one id, one role delta, one [DONE]; the finish, usage and [DONE] frames are held until the turn is settled, the tool-call frames while a check can act); the C4 repair of almost-JSON arguments; the tool-argument guard on the streamed arguments (it stops the stream, sends the sentence and finish_reason: stop, and the session's next request carries the note); D1 and D2 on the streamed words (server/conductor/monitor/tier0_words.py), closed by a second request that continues the reasoning (continue_final_message) on the vllm and sglang kinds and journaled in shadow on every other.
• Switches per backend and per model, each group inherit, off, shadow or on, never above the mode. Defaults: everything on for a local engine; on a cloud API the packages and E1 off and the tool checks in shadow until a model is switched on, since each costs tokens on every turn or a second request.
• The word-level loop detector's numbers (tools/loop_words_fp.py over the opencode database's reasoning, 2026-10-04): 6 of 2,674 finished turns fire and 11 of 20 loops, at the strictest of 32 settings. The engine's own D1 acted on that traffic, so the 6 may include loops it closed. The block ships calibrated: false: D1 over the proxy stays in shadow until the Compare view's labels show its precision.
• The dashboard: a backend's harness mode and profile on its form; next to each model switch what it resolves to, one sentence on what it costs or loses on that backend, and what its extra requests cost over the last 24 hours (the class questions are ledger rows of the client harness, a second request's tokens and cost sit in its journal row); the Compare view's shadow precision per backend (contract: backend in the precision rows, optional).
• The standalone proxy serves the library (/v1/library), Laya (when the decider role names a Jev endpoint) and both under /s/<token>/, and the conductor's own routes. A classifier or narrator role may name a backend (backend:<id>/<model>). The monitor's policy ships inside the package as JSON for an installed proxy.
• A relay that fails in the proxy's own code now answers (500, or an error frame in a started stream); before, the client waited for its own timeout.
• For a public repository later: personal names and the agents' names are out of the code's comments and the page texts; the Laya kit's default address is http://127.0.0.1:8000.
• Measured against the live engine from the Mac (tools/proxy_harness_check.py, the engine as a tandemllm backend, raw at the engine): the class question answered by the engine through its grammar (class debug, stage qwen), the note added and replayed on the next turn at message 1 (about 2,510 tokens estimated), a title capped at 64 tokens with thinking off.
• Not verified: a real vLLM or SGLang behind the harness (the close by continue_final_message and the reasoning parsers' output after it are tested on the fake backend only), the cloud providers' answers to the class question and the effort switch (documented shapes, no paid call).
• Tests: tests/test_proxy_harness.py 15 (the fake backend: off byte for byte, the replay with the prefix cache, E1, the regenerate and the repair, the argument guard and its note, D1 shadow on OpenAI and closed on vLLM, the side cap and the compaction section, the Backends API, the library under a session address), tests/test_proxy_engine.py adds the harness inside the engine's server, tests/test_conductor_config.py the packaged policy copy; dashboard e2e/backends.spec.ts adds the switches' lines at 1440 and 390 px. tools/proxy_e2e.sh runs the harness tests and, with --engine-url, tools/proxy_harness_check.py.

vLLM and SGLang at L1 (proxy P2 and P3, branch proxy-p23)

On branch proxy-p15 (the host collector and the package, not released). New backend kinds vllm and sglang (the same lines as on branch proxy-p4, which adds their switches for P4). Dashboard contract 1.3, additive. Nothing changes for the engine's own model, and a server with --backends off loads none of it.

• The engines' own metrics (proxy/engines.py, docs/proxy.md). For a backend of kind vllm or sglang the proxy reads the engine's /metrics page every 15 s through a name table checked against vLLM v0.30.0 and SGLang v0.5.21: requests running and waiting, KV use, the decode rate, the prefix-cache hit rate, the engine's TTFT, the queue, prefill and decode p50 and p90 (vLLM; SGLang has the queue only), preemptions, and speculation (vLLM from its counters, with acceptance per draft position; SGLang from its own window gauges). Figures come from the first and the newest reading of the last 5 minutes and are null where the page has no number. A failed read waits 30, 60, then 120 s; SGLang without --enable-metrics answers 404, and the card says how to turn it on. Discovery every 10 minutes: vLLM /version, SGLang /server_info.
• On /metrics: the mapped families as qse_backend_* with a backend label, qse_backend_info{backend,kind,version} and qse_backend_scrape_age_seconds. The engine's own families never see a backend.
• Contract 1.3: a backend's status row (summary.live.backends[], system.proxy.backends[]) gains engine (the page's state, its age, the version, what /server_info says, the figures); null for the other kinds. scrape_age_s is filled.
• The dashboard: a card per vLLM or SGLang backend on Performance (from the qse_backend_* families of its own /metrics scrapes) and on System (from the engine block); the engine on a backend's page and one line on its card; vLLM and SGLang in Setup and in the kind list. lib/prom.ts derives the same figures as proxy/engines.py; a test checks both on the same pages.
• Per request: a reported reasoning count of 0 next to streamed reasoning is null (SGLang sends reasoning_tokens: 0 when it did not count). Cached tokens per request need vLLM's --enable-prompt-tokens-details or SGLang's --enable-cache-report; reasoning tokens need --reasoning-parser. docs/proxy.md has the table.
• Found in the v0.30.0 source: vLLM removed guided_choice and the other guided_* fields and answers 200 without the constraint, so the class question uses structured_outputs.choice; vLLM takes X-Request-Id as the request id (chatcmpl-<id>); its stream sends reasoning as delta.reasoning. SGLang's collectors moved to sglang/srt/observability/metrics_collector.py, /get_server_info gave way to /server_info, and its /health runs a one-token generation when idle, so the proxy never calls it.
• Mapping one page of about 500 lines takes about 2 ms on the Mac, on the poller's own thread; a relay thread never waits for it.
• Tests: tests/test_proxy_engines.py 16 (the tables on pages built by tools/proxy_fixture_pages.py from each release's own metric definitions and rendered by prometheus_client, hand-counted figures, the poller's 404, back-off and discovery, the per-request counts on streams shaped by each release's protocol classes, the standalone proxy in front of three fake engines against the contract); dashboard: vitest tests/engines.test.ts 7, e2e engines.spec.ts (chromium, webkit at 390 px). Not verified against a running vLLM or SGLang yet: tools/proxy_engines_check.sh (in tools/proxy_e2e.sh --box --engines) starts each with a small model beside the live engine when memory allows, one at a time under the box lock, and checks the adapters against it.

The host collector and the package (proxy P1.5 and P5, branch proxy-p15)

On local main 162cbbc (0.3.29-test, live as 97a0ef3). New server flag --host-collector on|off (on). New backend setting host (TANDEM_BACKEND_<ID>_HOST, _HOST_URL, _HOST_DCGM_URL; TANDEM_COLLECTOR_TOKEN). New pyproject.toml: pip install . gives the tandem-dash command without torch.

• The host collector (proxy/hostcollect.py, docs/proxy.md). The System view shows the machine behind each backend: memory (unified on a GB10, where the GPU has no memory figure of its own), CPU, GPUs (NVML, else nvidia-smi), disks and NVMe, network, and the process behind each backend's port with its children, RSS, CPU, threads, uptime, GPU memory and its command line (home as ~, secret values masked). This machine first, with the engine's own process; then each remote backend's machine through tandem-dash hostcollect on it (JSON and Prometheus text behind a bearer token) or its node_exporter and DCGM exporter pages (proxy/scrape.py; no per-process figures). Sampled only when the view asks: at most every 5 s, the process scan every 15 s. The engine's system.gpu comes from the collector when it has a GPU, so the view reads the GPU once, not twice.
• Contract 1.3 system.hosts (optional): an older server sends none, --host-collector off leaves it out.
• The package: distribution tandemllm (cryptography alone; the host extra adds psutil and nvidia-ml-py), the built page inside as proxy/dashboard, tandem-dash serve | check | secrets init | backends list | hostcollect | version. check reads the environment and a copy of the config database through the schema and writes nothing. VERSION 0.3.29-test is the package version 0.3.29.dev0.
• ops/Dockerfile.dash (python:3.12-slim, the wheel with the host extra, an unprivileged user, /data volume, health check) and ops/dash.env.example. Built in CI only; not built yet.
• Measured on the Spark against the live engine (tools/hostcollect_check.py --port 8000, CPU only): MemTotal and MemAvailable equal /proc/meminfo to the byte; temperature, power, SM clock and use equal nvidia-smi; the engine's process found behind :8000 with 81,693,556,736 bytes of GPU memory against nvidia-smi --query-compute-apps' 81,693,507,584. One sample with a process scan costs 7.46 ms of CPU: 0.050 % of a core at the real cadence (scan every 15 s), 0.149 % if every 5 s sample scanned. On the Mac: 4.4 ms.
• Two machines: tandem-dash hostcollect on the Spark and a standalone proxy on the Mac with the live engine as a backend: the Mac's System view showed the Spark's unified memory, the GB10 and the engine's process with its GPU memory.
• tools/package_check.sh on the Mac: an 824 KB wheel, 7 packages in a clean venv, no torch, check and serve against a fake backend pass.
• CPU suite on the box: 186 of 186 files.
• Not verified: the per-process GPU memory on a discrete GPU (tested on a recorded NVML answer; no such machine here), the node_exporter and DCGM path against real exporters (recorded pages), the docker image.
• Tests: tests/test_hostcollect.py 12, tests/test_package.py 4; dashboard: vitest tests/hosts.test.ts 4, e2e system.spec.ts (the hosts at 1440 and 390 px). The night pass (tools/proxy_e2e.sh) runs them with tools/package_check.sh and, on the box, tools/hostcollect_check.py.

0.3.29-test, 2026-10-04

d2d91f3 (0.3.28-test, with the voice fix caed7b8) and the proxy, P1 (branch proxy-p1): the dashboard in front of other engines and cloud APIs. New server flag --backends PATH|off (off); ops/start.sh passes serve.env's QSE_BACKENDS (empty: off). Usage ledger schema version 3 (columns added in place at start). Dashboard contract 1.3, additive. With --backends off, as on :8000 after this deploy, the engine loads none of it and serves exactly as before; Qwen's text does not change (identity gate).

• The proxy (proxy/, docs/proxy.md). Requests are routed by model id to the backend that serves it: another TandemLLM, vLLM, SGLang, any OpenAI-compatible server, OpenAI (chat, completions, embeddings, and the Responses API relayed as it is) or DeepSeek. The answer's bytes go to the client unchanged; the proxy asks a stream for its usage frame when the client did not and drops that frame. /v1/models lists each backend's real model ids with owned_by its backend; two backends with one id are listed as <backend>/<model>. Routing rules (ordered globs), fallbacks before the first byte (connect error, 429, 502, 503; at most 2 tries again; Retry-After honoured up to 30 s), max_inflight per backend (64, then 503), timeouts per backend, a client that leaves closes the upstream connection.
• Two entry points: the engine's own server with --backends (a relayed request runs on its handler's thread and never touches the engine's lock or queue; TandemLLM stays the engine's), and python -m proxy with no torch and no GPU (Python 3.11+, the standard library and cryptography).
• The config database (~/.tandemllm/config.sqlite3): backends, keys as AES-256-GCM ciphertext (the master key from TANDEM_SECRETS_KEY, the OS keychain or a 0600 key file), models with public ids, prices and per-model switches, routing rules, users, an audit table. TANDEM_* variables overlay it and win field by field; a field a variable sets is locked in the GUI and a save that changes it answers 422 naming the variable; an env key is never written. One schema (proxy/config.schema.json) for both ways in.
• Measured, not estimated: tokens from the backend's usage (null without it), cached and reasoning tokens from each provider's fields, TTFT at the proxy, decode tok/s by the engine's formula over the token frames (null without usage: a frame is not a token), cost at the model's price when the row is written, the proxy's own time. A relayed row has null queue, prefill, blocks, draft counts and cache source. DeepSeek's off-peak half price applies by the request's UTC hour. proxy/prices.seed.json: OpenAI's and DeepSeek's list prices as read on 2026-10-04, offered on a model's row, applied only by a click.
• Users and keys: with one user, the OpenAI routes need a key from any other machine (the engine's own model included); trusted-local requests pass, so the conductor's own questions to Qwen keep working. The key's label is the ledger's user.
• The dashboard: a Backends tab (cards with health, models, in flight, latency, cost today; add after a test connection, edit, delete; the write-only key with its last 4 characters; env locks; models with prices and switches; routing rules; users with each key shown once) and a Setup screen while no backend exists. Usage filters by backend (engine for the engine's own rows) and by user and shows the cost of priced models; Dev and Performance tag relayed rows and say "measured at the proxy" and "not reported by this backend". At 390 px the bottom bar shows short labels (Perf, Play, Cond) so seven tabs fit.
• The proxy's own families on /metrics (qse_proxy_requests_total{backend,kind,finish_reason}, qse_proxy_tokens_total, qse_proxy_cost_micros_total, qse_upstream_retries_total, qse_proxy_overhead_ms, qse_proxy_ttft_seconds, qse_backend_up, qse_backend_inflight). The engine's families keep their labels and never count a relayed request.
• Measured on the Mac against the fake backend (tools/proxy_bench.py, 200 requests): the proxy adds 0.33 ms to the client's TTFT at the median, its own proxy_ms 0.17 ms p50 and 0.22 ms p90; 64 streams at once reached the backend all at once and ended in 5.18 s against 5.22 s for one alone. Against the live engine on :8000 from the Mac over the LAN (tools/proxy_gate.py, 20 raw requests, greedy, thinking off): token counts equal, decode within 0.8 % of the engine's own predicted_per_second, proxy_ms 0.37 ms p50, TTFT 11 ms above the engine's at the median.
• Live on :8000 since 19:55 on 2026-10-04 (97a0ef3, --backends off: the service serves as 0.3.28 did), through the drain; down 19:46 to 19:55. Gate identity PASS against c58862d; row with --no-learn-cost 50.09 tok/s (c58862d 50.08 at 19:21 in the 0.3.28 hold, the same flags); CPU suite 184/184; the ledger migrated to schema 3 with all 11,404 rows; smoke raw 27/27, on 27/27; 0 tracebacks. caed7b8 (the voice's language and gerund fixes) went live with it. Rollback: bash ~/rollback-0.3.29.sh on the box (to c58862d; the ledger keeps its added columns). The proxy on :8000: set QSE_BACKENDS=${HOME}/.tandemllm/config.sqlite3 in ops/serve.env and restart.
• Not verified with a real key: OpenAI and DeepSeek, which are tested on frames written from their documented shapes (tests/fixtures/proxy/frames.json), not on paid calls.
• Tests: tests/test_proxy_config.py 7, test_proxy_adapters.py 7, test_proxy_relay.py 10 (the fake backend, tests/fake_backend.py), test_proxy_engine.py 2 (the engine with --backends, the fake engine); dashboard: vitest tests/backends.test.ts 7, Playwright e2e/backends.spec.ts on chromium desktop and webkit at 390 px. tools/proxy_e2e.sh runs them all together with the bench and the gate.

0.3.28-test, 2026-10-04

10c2021 (be4848e, 0.3.27-test, with the lent_ms fix 2a9e66b) and branch tandem-n3: narrator v3, the voice. New profile value ui.narrator: live and block ui.live (absent: as before). The deploy names qwen3.5-2b as the narrator's model in serve.env (the shadow reader keeps qwen3.5-0.8b in a slot of its own) and stores coding v18 (ui.narrator: live) and chat v3 (ui.states: on in the reasoning, narrator: live, the router's and the packages' lines off). Qwen's text does not change (identity gate).

• The voice (server/conductor/voice.py, docs/server.md). One line of what Qwen is doing right now, in the profile's channel (opencode: a titled reasoning part a line): a thinking line every 25 s from 15 s ("thinking 41 s: making pits narrower to widen the jump window"), what a thought of at least 10 s decided when it ends ("thought 25 s: fix the shader bug by using instanceMatrix[3][2] instead of [4]"), what a long write is for and how far it is ("writing rrr4/index.html: display the pagoda garden with a fixed HUD", "writing render.js, 228 lines so far" every 30 s), a plan in the answer. The engine writes the facts (the timer, a call's action and target from its open block, a write's lines); the small model only the phrase after them, from a window of Qwen's newest ids, and every phrase is checked: a name or number it holds must be in the text it read, most content words too, no first person, no copy of an example, the user's language, nothing like the last four lines. Nothing for a short turn (no line before 15 s of thinking, no tool line for a call done within 1.5 s).
• opencode's write writes content before filePath: a write's line names the file only when the body's header comment does ("// render.js: ...") or its first line names the one file Qwen's words just before the call name; else "writing a file".
• The harness's lines in plain words and the user's language (German or English): "code edit, maximum effort", "packages coding, testing, laya-testing, laya-playtest and 2 more" (one line for a turn's packages), "waiting for another request to finish, 1st in line", "reading 20,745 tokens, about 18 s left". The templates stay for every profile without live.
• One share of Qwen's time per request for the small model (narrator.Share, 3 % of the time since Qwen's first token): the voice first, the shadow reader's in-turn reading after it, at most its 2 %.
• The narrator's model loads at the start (ops/start.sh asks after the decision models; the keeper loaded the 2B in 18.5 s, 3.71 GiB of MemAvailable) and stays loaded (a keeper thread loads it again in an idle window). Before, it loaded at its first line, copying its batches under the engine lock between a running turn's rounds: 734.5 s on 2026-10-04.
• The dashboard: the newest line on the Performance view's Live card and in each request's row (the live view's line, contract 1.1, optional).
• Measured: the voice's decode A/B on the candidate (tools/thought_narrator_measure.py --part impact --on-mode live, the 2B, --no-learn-cost, the shadow off, 2 warm-ups and 3 runs an arm, the same text in every run, 504 blocks): off 63.47 tok/s, on 62.26 tok/s, -1.90 % (gate 3 %; spread 0.26 % and 0.19 %), four lines a run. The 2B's time per phrase on real sessions (tools/voice_replay.py on the GPU, 4 opencode sessions, 274 phrases): p50 308 to 369 ms, p90 431 to 597 ms, in slices of at most 150 ms; the 0.8B 153 to 155 ms p50 with clearly weaker lines. Row with --no-learn-cost: c58862d 50.08 tok/s (be4848e 50.88 in the 0.3.27 hold). Gate identity PASS against be4848e; CPU suite 180/180; smoke: a coding request through :8000 got 8 lines (route, packages, a plan, three thinking lines, the thought, the write).
• Live on :8000 since 19:24 on 2026-10-04 (c58862d; down 19:01 to 19:13 for the gate and the A/B, 19:15 to 19:23 for the row and the deploy). Rollback: bash ~/rollback-0.3.28.sh on the box (code, serve.env and the two profiles back to be4848e's). The voice alone off, no restart: bash ~/n3/voice-off.sh on the box (coding back to narrator v2, chat as before), or per profile ui.narrator: thoughts|off, or for all the roles document's narrator.mode: off.
• Tests: tests/test_voice.py 15 (the facts in words, the tool watch, the clean, thinking, tool and plan lines on a fake clock, the share, the handler: the lines in opencode's inline channel, never in Qwen's next prompt, German, off byte for byte); opencode 1.18.34's own parser (node, tools/opencode_parts.mjs) on the stream: one titled reasoning part a line, a live write's lines after its tool part. tools/voice_replay.py replays a real opencode session through the voice's code.
• After the deploy (on main, not deployed; it goes out with the next restart): an English user's phrase in German is refused (the smoke's plan line came out German), a thinking line in the third person reads as a gerund.

0.3.27-test, 2026-10-04

aca8126 (0.3.26-test) with branch tandem-d8 and 854aae9. New server flags --side-lane on|decode|off (on), --side-lane-prompt-max (2,048), --side-lane-new-max (64); a lane column in the usage ledger (added in place at start). Every request the side lane does not serve gets the same text as on 0.3.26; the long request a guest interrupts gets the same text as without it (measured below).

• The side lane (server/preempt.py, docs/server.md). On 2026-10-04 between 16:43 and 16:48 one opencode turn thought for minutes, and everything short waited behind it in the single queue. The anon requests the ledger shows as abandoned after 9.0 s were the router's own package and library questions to Qwen (server/conductor/classify.py, raw, 24 tokens under a choice grammar, an 8 s timeout) for a second opencode session's first turn; that session's turns got a 429 after 256 s. Now a short request -- a raw one or a client's side request (a title), thinking off, at most 2,048 prompt and 64 output tokens -- that finds the engine busy is served by the running generation at its next safe point, between two decode blocks or two prefill chunks. The guest is prefilled from position 0 and decoded one token a forward without the drafter, so its answer is plain greedy decoding of its prompt.
• The generation gets the engine back byte for byte (engine/cache.py Borrow): the KV rows the guest can reach, the recurrent state, the convolution tails, a walked chain state and the pending commit (copied unsettled, so the next verify folds the same record), and the references the next block reads; the drafter is never touched. A job waits for a later safe point when it would push the engine past its memory peak; a disk-tier write that has not copied the guest's rows yet stops where it is; a job nobody takes runs under the lock on its own thread. A client's side request too long for the lane (opencode's summary) is never refused for its wait.
• What it costs and what it gives, on the real engine (tools/lane_check.py, the candidate with every cache off and --no-learn-cost, a 5,000-token prompt and 768 output tokens with thinking, greedy): with 10 short requests fired into it (opencode's title shape and the router's question shape), the long request's reasoning and answer were byte-identical to two runs alone. 9 of them were served in the lane, 1.2 s at the median and 5.1 s at most (two arrived during a prefill that ran as one chunk on that server, so they waited for its first decode block); the one over the lane's output cap waited 23.7 s in the queue, the rest of the long request. The long request took 7.4 s longer (32.9 s against 25.5 s) for 4.7 s the guests held the engine. Live on :8000 after the deploy, during a 2,094-token opencode-shaped turn: two title requests in 2.6 s (one served between two prefill chunks) and 1.6 s; a second session's first turn had its router questions answered by Qwen in 0.64 to 0.81 s each (test-lib at 0.998) where they had timed out at 8 s, and its turn itself waited in the queue as before. Not measured on the GPU: a guest between two prefill chunks against an uninterrupted prefill (the CPU test covers it bit for bit).
• Live on :8000 since 18:32 on 2026-10-04 (be4848e), through the drain. Gate identity PASS against c957346; rows with --no-learn-cost: c957346 45.42 tok/s (disturbed by other CPU jobs on the box; the same tree read 50.34 at 17:24), be4848e 50.88; smoke raw 27/27, on 26/26, 0 tracebacks. Rollback: bash ~/rollback-0.3.27.sh on the box (to c957346).
• After the deploy (2a9e66b, goes live with the next release): the live check's first title line said lent_ms=0: a stream ends at its end token, before the engine is given back, so the request's line was written first. The request now waits the few ms for the give-back.
• Replays are no speed sample anywhere (the owner, 16:51: a dot at 160,500 tok/s flattened Performance → Recent requests). The dot was the 13:08 compaction answered from a summary written ahead (cache_source: prepared), which every filter missed: they knew only response. Both kinds are now left out of the speed percentiles, the Prometheus histograms, the live rates and the scatter; the ledger records no decode rate for a replay and the API shows none for older rows either. The scatter lists replays with their TTFT under its switch; its y axis ends at twice the 95th percentile, and a point above it is a triangle at the top edge. Side-lane rows are no speed sample either, and carry a "side lane" tag in the live panel, the recent table, the timing breakdown and the System view's queue card.
• The shadow's pool release (0.3.26) parked two sessions on the disk in 10.4 s under the engine lock. It now yields to a request that waits, between sessions; the parked ones stay on the disk and the next idle burst goes on.
• The tool-argument loop guard also stops a call that streams live (854aae9). On 0.3.26 it acted only on held calls, and opencode's coding profile holds none (its tool gate is off), so the live check logged the loop at 1,088 tokens and Qwen ran on to 65,536. Now the generation stops there as well; the block's text is never sent again as content (the token limit used to send the whole cut block a second time, 1.26 MB on 2026-10-04); the client keeps the call it was sent part of, which opencode runs as an invalid tool, so the next step comes at once with the hidden note. Tests: tests/test_argloop.py 11 (a live-streamed call stopped, its note on the next turn), test_toolcheck 23, test_toolcall 45, test_fake_engine 10, test_empty_turn 10, test_app_loop 29 on the box CPU.
• Tests (box CPU): tests/test_side_lane.py 19 (the engine bit for bit after guests between blocks, after a rollback, two in a row, between prefill chunks; the guest is plain greedy; the lane's waits, withdrawals, failures and memory room; the disk guard; the handler with the fake engine: a title during a long turn in under 2 s with the turn's text unchanged, the router's question in the lane, what stays out of it, a long side request past the queue timeout); tests/test_shadow.py 1 new; the dashboard's tsc, vitest (27 files, 349 tests) and the Performance and System e2e on desktop and at 390 px. The CPU suite 179 of 179 files.

0.3.26-test, 2026-10-04

5cd7e95 (0.3.25-test) with branch tandem-d7: two live bugs of the afternoon. One new server flag (--argloop on|log|off, on by default) and one serve.env key (QWEN38_SHADOW_POOL_SLACK_GIB, default 2). Every request that does not loop inside a tool call gets the same text as on 0.3.25.

• The tool-argument loop guard (server/argloop.py, docs/server.md). At 16:18 Qwen wrote the same 97-character tile-map row 12,866 times inside a write call: 65,536 tokens, 327 s, nothing written, and the cut text made the next prompt 157,806 tokens. The guard reads the open call's arguments every 64 tokens and stops a held call when its last 1,024 tokens are one unit of at most 4,096 characters repeated 6 or more times, or when it passed 32,768 tokens and 3 times the request's longest message with its last 16 KB compressing 16 times or more. The client gets one sentence and finish_reason: "stop", no call; the session's next request with tools carries a hidden note (write it without the repetition, generate repeated rows with code, write a large file in parts). On the incident's text it fires 2,176 tokens into the call, 1,055 after the loop began.
• False positives: none on the 4,748 writes and edits of the opencode database (tools/argloop_fp.py; 517 over 1,024 tokens, the largest 31,137). The longest exactly periodic stretch in them is 307 tokens, the highest window ratio 25.4 (a 21,256-token file, under the size floor). A call streamed live (no acting monitor) is only logged.
• max_tokens 65,536 from opencode stays: no per-call cap (the largest legit write is 31,137 tokens). D1 and D2 act inside the reasoning only, so long legit answers and calls never meet them.
• The memory gate refused every request after the shadow reader followed a 195,694-token conversation (until the restart at 16:41, 12-token prompts included: "needs 82.0 GiB, peak 77.0 GiB"). Its model's pool held 17.6 GiB of free segments: a live torch MemPool never gives them back (measured on torch 2.13: empty_cache returned 556 freed MiB only after the pool was deleted), and every KV growth and every chunk's attention temporaries took a new one. After a burst with more than 2 GiB free in the pool the shadow parks its sessions on the disk and unloads the model, which drops the pool; the gate has a last level, roles, that does the same; before a refusal the gate collects and empties once more; a prompt whose workspace fits in the engine allocator's own free reserve is admitted (stale_admit); a refusal is an error line in the Dev tab. The abandoned prefill was not the cause: the shadow read ten bursts after it and the refusals began right after them.
• Live on :8000 since 17:24 on 2026-10-04 (c957346; :8000 down 17:08 to 17:24 through the drain). Gate identity PASS against 5cd7e95; one row each with --no-learn-cost: 5cd7e95 49.14, c957346 50.34 tok/s. The shadow released its pool once in the first hour of a user's session (5.14 GiB reserved, 3.05 GiB in use, 2 sessions parked, 10.4 s of parking under the engine lock); no memory refusal since the start.
• Found live: the guard sees the loop but does not stop it for opencode. The coding profile's tool gate is off (C1 to C4 off), so no call is held and every call streams live; the guard only logged. The check (tools/argloop_live.py, a 5,251-token prompt asking for 400 identical map rows) logged would stop write: rule=periodic tokens=1088 period=98 and Qwen looped to 65,536 tokens (367 s) anyway; one other request left the queue after 9 s while it ran, and the tool's second turn sent the cut text back as content: a 70,804-token prompt, 170 s of prefill. Fixed in 0.3.27-test.
• Tests (box CPU): tests/test_argloop.py 10 (the incident's shape fires soon after the loop starts, legit shapes do not, the size rule, the handler with the fake engine: the stop, the next turn's note once, a long legit write whole, log-only without held calls, off and log); tests/test_budget.py 3 new (the roles level, the stale reserve, a role's free reserve is not the engine's); tests/test_shadow.py 1 new (released past the slack, the session back from the disk, bit exact).

0.3.25-test, 2026-10-04

ce78689 (0.3.24-test) with branch tandem-sr1: the shadow reader. New serve.env keys QWEN38_SHADOW_MODEL, QWEN38_SHADOW_DIR, QWEN38_SHADOW_GB, QWEN38_SHADOW_GRID, QWEN38_SHADOW_DISK, QWEN38_SHADOW_DISK_GB (empty model: off, nothing imported); a profile's shadow block (absent: off). The deploy sets QWEN38_SHADOW_MODEL=qwen3.5-0.8b and QWEN38_SHADOW_DISK=auto and stores the coding profile with shadow: {mode: on, router: shadow, router_min_tokens: 8192, narrator: off}. Qwen's answers do not change.

• The shadow reader (engine/shadow.py, server/conductor/shadow.py, docs/shadow-reader.md). Qwen3.5-0.8B keeps its own KV and Gated-DeltaNet state of each coding session, keyed by the router's session key, and reads each turn's new tokens: the served prompt and Qwen's output, as Qwen's own token ids (the Qwen3.5 small models share Qwen3.8's vocabulary and merges). It replaces the two closed-form bridges that failed earlier today (B2's soft rows at chance, B3's KV/GDN map at R² 0.5).
• Exact by construction: the state advances in whole 512-row chunks at fixed positions, so it equals a fresh prefill of the same ids on the same grid bit for bit (measured on the GPU on two real conversations of 41,802 and 58,278 tokens: every K and V row, the recurrent and the convolution state). Against a fresh prefill on a 2,048-row grid: 8 of 8 question probes give the same top token, KL at most 4.1e-5. A prompt that differs from what was read goes back to the newest checkpoint at or below the difference (one at each turn's prompt end).
• When it reads: in the idle window (every 0.2 s, behind every waiter and the summary written ahead, under the box lock taken without waiting, at most 2 s, stopping before the next chunk when a request waits: one chunk is 36 to 50 ms), and inside a turn in 2 % of the time since Qwen's first token, between two rounds. The idle window alone is not enough for opencode: the ledger's 41 sessions (1,624 turns) have a median gap of 0.31 s between requests; replayed with the measured rate the shadow is fresh at 41 % of turn starts idle-only and at 84 % with the in-turn share.
• What it costs Qwen (tools/shadow_ab.py, a hold, 6 runs an arm, a 16,350-token prompt): decode 36.02 tok/s off, 35.28 on (-2.05 %, gate 3 %); TTFT unchanged (17.46 / 17.40 s). The first build counted the share from the turn's start and lost 5.3 %; fixed before this release.
• Cost and memory (tools/shadow_measure.py, 0.8B): 71 to 97 ms per 1,000 tokens read at depths 0 to 41k (2B: 99 to 135 ms); 12 KiB of KV a token plus 19.5 MB of recurrent state: a 58k session is 0.88 GiB. Resident states are capped at 4 GiB; past it the least recently used session goes to ~/.qwen38-spark-engine/cache/shadow (20 GB, 48 h) and comes back exact; a planned stop writes them all.
• ask(session, question, choices) and POST /v1/conductor/shadow/ask: a typed decision of the small model over the whole session, one forward (68 ms for the router's 9-class question at the end of a 41k session). The router's class question is asked of the shadow in shadow mode on turns of at least 8,192 tokens and journaled beside the router's class in <conductor root>/shadow.jsonl (agreement, how far behind the shadow was when the turn began); nothing of the turn changes. GET /v1/conductor/shadow/status.
• The narrator can read its thought summaries from the shadow (shadow.narrator: on), whole session instead of the thought's last 1,500 ids; shipped off (its fork decodes eagerly, 12 to 17 ms a token on the 0.8B).
• Live on :8000 since 16:05 on 2026-10-04 (5cd7e95). A 5-turn test session of opencode's shape (tools/shadow_live.py, bearer sr1-test, a 12k-token system text, 12,676 to 21,491-token prompts): the shadow followed every turn (3 rollbacks to a prompt-end checkpoint when an answer came back re-rendered, 1,176 ids read again; 24,064 rows in 47 chunks, 1.65 s in all), 83 ids behind at the end, 0.38 GB. The router's question was asked on all 5 turns (77 to 124 ms each) and agreed with the router 0 of 5 times: the 0.8B picked debug 4 times and design_ui once, at 0.15 to 0.50, where the router said chat or code_edit at 0.72 to 0.997. That is the shadow-mode result the switch exists for; the 0.8B is not a router as asked. POST /v1/conductor/shadow/ask on the same session: code at 0.69 in 24 ms. Smoke raw 27/27, on 26/26, 0 tracebacks. Gate identity PASS against ce78689; one row each with --no-learn-cost, ce78689 49.73 and 5cd7e95 50.68 tok/s.
• The narrator's summaries read from the shadow, on that session's three thoughts of 128 tokens or more (offline, the same ids): "thought the parse_size function needed a docstring and doctests to clarify its behavior; it decided to add them and created the file", 433 to 678 ms each (eager decode). shadow.narrator stays off.
• Narrator v2 is on for opencode since 0.3.24 (D6, -2.5 %); with the shadow's 2 % share the two together cost Qwen's decode about 4.5 % on a coding turn that thinks. Not measured together.
• Rollback: bash ~/rollback-0.3.25.sh on the box (to ce78689, 0.3.24-test: code, serve.env without the QWEN38_SHADOW_* keys, profiles/coding v16).
• Tests (box CPU): tests/test_shadow.py 13 (exactness turn by turn, forks, rollback, disk, the service, the router question, the cap, ask, the narrator job, the in-turn share, the handler byte for byte); the CPU suite 177 of 177 files.

0.3.24-test, 2026-10-04

f0e4931 (0.3.23-test) with branch tandem-d6, live on :8000 since 15:17 on 2026-10-04 (ce78689): the end anchor survives an eviction after a restart, a breakdown of every spill save, the narrator's A/B made repeatable, narrator v2 on for opencode, the Laya drive kit closes its browsers, and the summary written ahead gives the box back. One new server flag (--learn-cost, on by default, as before). The changed engine files are engine/cache.py and engine/spill.py, which the disk tier's and the spill's fingerprints leave out, so the deploy kept both (26 tier entries, 6 conversations). After the deploy: serve.env QWEN38_NARRATOR_MODEL=qwen3.5-0.8b (was minicpm5-2b) and coding v16 (ui.narrator: thoughts).

• The end anchor through an eviction. On 0.3.23 (13:34) the conversation read back at the start had its end anchor, and the next eviction wrote it to the NVMe without it (end=False, 0 bytes written); its next turn would have re-read the 759 tokens above its last grid anchor. The request that evicted it was served from the disk tier, and prefill dropped the end before begin, whose eviction then saved the rows without it and deleted the old end.anc. The drop now comes after begin. A save that finds every block and the tokens unchanged also keeps the end anchor already on the disk when the engine no longer holds one. tests/test_session_spill.py reproduces the live case (fails on 0.3.23, passes now) and the carried end. Live after a restart (tools/spill_live.py --phase endcheck): another session's 11,130-token request served from the disk tier (10,240 reused) evicted the 41,803-token conversation with its end anchor (end=True, 0.24 GB written in 187 ms), and the conversation's next turn reused 41,803 of 41,822 tokens.
• The slow eviction (7.0 GB in 4.5 s right after the disk tier wrote 17.6 GB) is the NVMe's write cache. tools/spill_io_probe.py measures it: 4-5.6 GB/s for the first 16 to 24 GB, then 0.3 to 1.3 GB/s, back to 5.2 GB/s after 300 s of rest. The real save of a 58,278-token conversation from GPU buffers took 1.46 s on a rested drive and 4.9 s after 17.5 GB of tier-shaped files. Pacing the tier's files did not help (8.2 s), so the tier is unchanged. At the slowest rate measured (0.36 GB/s) the save takes about 20 s, about a third of re-reading those tokens (about 57 s). Every save now logs its parts in ms (lock, staging, sums, copy, waits for the writer, anchors, fsync, manifest) and writer_ms.
• Repeatable speed runs: --no-learn-cost keeps the length router's served price table. With learning on, the same 2,000 greedy tokens took 366, 468 and 504 blocks in 0.3.23's narrator A/B at 93.5 to 94.1 ms a block, because the router learns verify and draft times across requests and then chooses differently; that was the 37 % spread of the off runs. The narrator measure runs its server with the flag, sets a progress mark every 512 thinking tokens on both sides (the 0.3.23 runs never closed their thought, so the narrator did nothing until the turn's end), records blocks, ms a block and the text of each run, and reads its 3 % gate only when the off runs agree within 3 %.
• Narrator v2 on for opencode (coding v16, Qwen3.5-0.8B). The A/B with the model loaded, the router's costs held and a progress mark every 512 thinking tokens (three summaries in a 2,000-token run, more than the default 4,096-token mark gives): on the six runs that wrote the same text in 514 blocks, off 41.69 tok/s (spread 0.19 %), on 40.64 tok/s (spread 0.17 %), -2.5 %, inside the 3 % gate; 95.7 against 93.3 ms a block. Each turn ends about 485 ms later (the summary's end allowance). Live: Thought 7s: would use a fixed-timestep accumulator loop ... on an opencode-shaped request at 58 tok/s. The first ask after a start can be refused while the Laya models load (refused_load_in_progress); the slot asks again after 60 s and loads in 13 s (1.74 GiB). Undo: ~/c324-narr-off.sh on the box.
• The Laya drive kit (server/laya_kit/laya_drive.py) closes what it starts. 42 Chrome processes of four drives started a day earlier ran on the box with their driver gone; they were killed. Chrome and the game now run in their own process group, ended whole; a page that never comes up closes its Chrome; SIGTERM and SIGHUP run the cleanup; on Linux Chrome gets the parent-death signal, so a driver killed by SIGKILL takes it along. A kit copied into a project's .tandem/ keeps the old code until it is copied again.
• The summary written ahead gives the box back: it does not start while another program is on the GPU, and stops when a process waits in flock for the box lock it holds (/proc/locks, every 0.5 s) or another program comes to the GPU (nvidia-smi, every 5 s). A hold then waits about half a second instead of up to 45 s. prepared.stats.yielded counts it.
• tools/spill_live.py --phase endcheck: the live check of the end anchor after a restart.
• Not built (decided): a keep-alive for long non-streamed requests, the 0.3.21-era open item. In 14 days of the ledger no non-streamed request ran over 158 s, 3 over 60 s; opencode and Hermes stream.
• Checks: CPU suite on the box 176 of 176 files; gate identity PASS against f0e4931; one row each with --no-learn-cost, f0e4931 50.21 and ce78689 49.37 tok/s (-1.7 %; 4.65 and 4.63 tokens a block); smoke raw 27/27, on 26/26; the drained conversation read back at each restart (41,720 tokens, 5.21 GB in 1.18 s, end anchor on); 0 tracebacks. Rollback: bash ~/rollback-0.3.24.sh on the box (to f0e4931, serve.env and coding v15 as before the deploy).

0.3.23-test, 2026-10-04

83674ea (0.3.22-test) with branch tandem-d5: a client's compaction served from the conversation the engine already holds, a summary written ahead, a session token on the addresses the notes give the model, the session spill's own I/O path, and the skill canon for every location. New flags --compaction, --compaction-prepare, --compaction-idle-s, --compaction-min-tokens (serve.env QSE_COMPACTION*, absent = continue, prepare on, 30 s, 32,768 tokens). No conductor document changes.

• Compaction as a continuation (server/compaction.py, docs/caches.md "Compaction as a continuation"). On 2026-10-03 a manual /compact of a 111,636-token opencode conversation read 63,614 tokens cold and took 189.8 s, and an automatic one at 198,030 tokens read 129,643 tokens and took 270 s, both while the conversation sat resident. opencode sends its compaction agent's prompt and ONE user message holding the history as text, which shares no token with what the engine read. Now the request is served as the session's own conversation (up to the end anchor of its last generation, in RAM or on the NVMe) plus one user turn with the client's system text and instructions: the prefill reads that turn alone. A guard checks the first and last user lines of the client's history against the record. The end anchor after images is kept with its rotary offset, for this continuation alone.
• A summary written ahead. When the engine has been idle 30 s, the resident conversation of an opencode session of at least 32,768 tokens gets its summary written in scratch rows, under the box lock taken without waiting; a waiting request or a drain stops it at the next chunk or block and the conversation's rows, anchors and end anchor stay as they were. A compaction of that session at that end anchor with the same instructions is answered from it without the engine.
• Measured live (2026-10-04, a captured rrr6 request from a user report, 104,313-token turn with 17 images, 57,008-token compaction request): served as sent (cold) 115.5 s (80.8 s of prefill); as a continuation 40.7 s on f0e4931 (prefill 1.04 s, 56,551 of 57,008 reported cached; the rest is writing 1,406 summary tokens); from the summary written ahead 0.07 s (written 75 s after the turn, in 43.9 s). Every summary kept opencode's eight template sections. On 0.3.20 the same compaction had taken 189.8 s.
• Session addresses (server/session_tokens.py, docs/server.md "Session addresses"): {tandem_url} in a session's notes is the request's address plus /s/<token> (12 random characters, kept 48 h, 0600), so a library or Laya call from the model's scripts names its session in the journal row and the session's counts; an unknown token is served, attributed to nobody; other routes under /s/ are a 404. A client without a session id keeps its session through a compaction the engine wrote (a new head carrying that summary).
• The session spill's own I/O path: its own lock and staging buffers, the disk tier's writer waits between files while a save runs, a block's copy overlaps the previous block's write, one device sync for a block's sums, unchanged anchors not written again. Live, the same case as 0.3.22's (a 41,587-token conversation evicted by another session's 9,162-token request): 5.20 GB written in 1.03 s (0.3.22: 7.23 s), read back in 0.94 s, the next turn's TTFT 2.0 s. A 58,278-token conversation evicted right after the tier had written 17.6 GB took 4.5 s (7.0 GB); why it was slower is not measured.
• The skill canon for every location (server/skillcanon.py). Four first requests of fresh sessions captured from two opencode 1.18.34 processes still parted at character 24,970: two skills had their second copy under ~/.claude/skills/.trash/, which 0.3.22 did not touch. Every entry with a path is canonical by name now. Live: the 2nd to 4th request reused 37,888 of 38,172 tokens (the first read all 38,172 in 54.3 s).
• The disk tier's fingerprint leaves out the files that keep state (engine/cache.py, disk_cache.py, spill.py, capture_lock.py), as the spill's does: 0.3.22's deploy swept 49 GB of heads for a change in cache.py alone. This deploy sweeps once more.
• Narrator v2 stays off. The A/B now runs from a tree without the hold's pause file, so the model really loaded (1.89 GiB, 12.5 s); 0.3.22's -14.6 % was measured with the model refused (refused_hold, the hold's own .watchdog.off in the served tree) and so was not the narrator's cost. This time, 2,000 tokens a run: off 58.1, 45.6 and 42.2 tok/s, on 45.6, 45.6 and 45.7, with about 480 ms at the end of each turn on (the end allowance), and no thought summary in the on runs (one 2,000-token thought, still open at the cap). The off runs alone spread 37 %, so this A/B cannot show a 3 % difference either way. Summaries alone: 11 of 12 written, p50 395 ms, p90 499 ms; the graph and transformers' greedy agree.
• Fix found live on the first build (1aa4d99): a continuation after images carried the images' key ids into the suffix store, whose packing raised after the summary, and the stream lost its usage chunk. Fixed in f0e4931.
• Tests (box CPU): test_compaction_continue 13, test_session_identity 5, test_session_spill 10, test_skillcanon 3; the CPU suite 176 files; dashboard tsc clean, vitest 344 passed. Gate: identity PASS; one row each, 83674ea 50.28 and 1aa4d99 51.25 tok/s.

0.3.22-test, 2026-10-04

40a043d (0.3.21-test) with the paused work of branch tandem-d3 and the narrator of branch tandem-n2 merged (branch tandem-d4), and four fixes from a day of user reports on the box. New serve.env key SESSION_SPILL (absent = on); ops/start.sh passes --conductor-defaults shipped to the :8000 service.

• Live conversations on the NVMe (engine/spill.py, docs/caches.md "Live conversations on the NVMe"). The 15:35 deploy cost the next turn of a 130,000-token opencode conversation a 158 s re-read: the disk tier keeps only the cold-grid head (25,600 tokens), and everything past the first answer is a session's rows. A planned stop (SIGTERM's drain, after the work in flight) now writes the resident conversation and the parked one, and the next start reads the resident one back before its first request: its rows of every buffer, the end anchor and the newest 4 anchors, its tokens, taint and session, the bytes that were in the engine. The next turn resumes at the end anchor.
• A long conversation is no longer evicted by another client's request. At 15:37 a 10,216-token request of another client (over the guest size, 8,192) replaced the resident prompt, the RAM park holds 35,000 rows above the shared head, and the conversation's next turn re-read everything above 32,768: 165 s. Such a request now writes a resident conversation of at least 16,384 tokens to the NVMe first when the park cannot hold it, and a session snapshot that clears the resident prefix does too; the conversation's next turn reads it back (cache_source: spill). The RAM cap and the 262,144-token window are unchanged.
• The spill writes and reads by content: blocks of 1,024 rows with two 64-bit sums per buffer computed on the device; a block whose tokens and sums equal the file is not written again, a block in place is not read, every block read is summed again before the engine resumes from it. Its own budget (40 GB), the tier's 50 GB floor, 48 h. Its fingerprint leaves out the files that keep state (engine/cache.py, disk_cache.py, spill.py, capture_lock.py). Flags --session-spill, --session-spill-gb, --session-spill-min-tokens.
• A client's effort word is a ceiling (a design decision, 2026-10-03 12:30): where the router decides the effort, the turn never gets a cap above the word's (medium 8,192); a number still wins outright, the answer reserve stays. Per profile thinking.client_word: ceiling (the default) or ignore; coding and overnight ignore it, because opencode sends medium on every turn. The deploy stores the live coding and overnight profiles with ignore.
• One location per skill in opencode's skills list (server/skillcanon.py). D3 found that every new opencode session re-read 5 to 6 thousand tokens of its head: 11 of the 32 entries of <available_skills> sit in two folders the Claude app syncs, and opencode keeps whichever copy its scan meets last, so their location flips between sessions. An opencode request (not raw) now gets each synced entry as the copy first seen for its name; the choice is kept in the conductor's directory.
• From branch tandem-d3 (2026-10-03): the end anchor for a copy whose whitespace is not the rows' (server/endalign.py, END_ALIGN): within a session some tool turns re-read 2 to 3 thousand tokens because opencode's copy of a tool call's parameter or of the </think> line differs from what the engine wrote in whitespace alone; such a copy now resumes at the end anchor. The Hermes agent is a client kind of its own (hermes, by its tools, its title call or its identity) and gets the hermes profile. The cache source of each request on the Live panel and the Performance screen. The live documents of 2026-10-03 shipped for a new box (schema.py SHIPPED_*, the texts under packages/), stored as version 1 at the first start by server/conductor/shipped.py for each document a box has no version of (on the reference box: the hermes and general profiles); the built-in version 0 stays the minimal one.
• From N2: the narrator sums up each thought (ui.narrator: thoughts, Qwen3.5-0.8B). Shipped off. On the GPU (2026-10-04) it wrote 11 summaries of 12 real thoughts, 442 ms at the median and 712 ms at p90 (a decode step 9.5 ms, the captured graph and transformers' greedy give the same ids); the decode gate did not pass: the candidate server's planner refused to load the model inside the hold (refused_hold), and the runs with the narrator profile were 14.6 % slower than without it all the same. Not switched on until a clean A/B with the model loaded passes the 3 % gate.
• Measured on the live box (2026-10-04): a 41,587-token conversation evicted by another session's 9,162-token request was written in 7.2 s (5.2 GB; the disk tier was writing the same conversation's head at the same time) and read back for its next turn in 1.3 s, TTFT 2.5 s where its cold first turn took 56.9 s. A restart through the drain wrote 0.85 GB (40 of 41 blocks unchanged) in 1.2 s, the start read 5.2 GB back in 1.1 s, and the next turn resumed at the end anchor: 41,719 tokens reused, TTFT 635 ms. tools/session_spill_check.py on the served engine: every row, the end anchor and the restored anchors equal bit for bit in the writing process and in a fresh one, the next turn's tokens equal. Row pair: 40a043d 44.48, 0.3.22 50.10 tok/s (one run each).
• Tests (box CPU): tests/test_session_spill.py 9, test_skillcanon 2, test_defaults 4, test_client_kinds 4, test_client_budgets (the ceiling), test_end_align, test_thoughts; the CPU suite; dashboard tsc clean, vitest 344 passed, dist 153.8 KB gzip of 300.

What changes on deploy: the server's and the engine's code (engine/cache.py, engine/disk_cache.py, engine/spill.py: the disk tier's fingerprint changes once, so its files are swept at the first start and its counts kept); the deploy's own restart cannot keep the resident conversation (the old process has no drain write), every later stop does.

0.3.21-test, 2026-10-03

A hotfix on 50cc1f5 (0.3.20-test, branch tandem-img1): a long conversation is no longer refused for the images it has gathered, and after a compaction the session's class packages come back. No serve.env key, no ops/ file, no engine file and no conductor document changes; a request whose images fit the caps and that is not a compaction or the first user message after one is served byte for byte as by 0.3.20-test.

• The bug (a user's opencode session in ~/rrr6, 2026-10-03): the design-review package (packages v6) has the model screenshot every section and read each png, and opencode sends every earlier image again on every turn. The 17th (several were re-reads of one file) took the request over --max-images (16): HTTP 400 17 images; the limit is 16 a request on messages[210].content[1].image_url, not retryable, and every later request carried 17 or more, so the session could never continue.
• server/images.py: when a request's images are over --max-images or --max-image-rows and the excess is in earlier messages, the oldest images are left out instead of a 400. A left-out image is never fetched or decoded; build_prompt renders it as the text [an earlier image was left out: TandemLLM keeps the newest 16 images of a conversation] (images.LEFT_OUT, one text for every left-out image of a server) in place of its placeholder, so the template writes no placeholder for it and the placeholders left match the images kept. The router, the session monitor, the outcomes and the transcripts read the client's messages unchanged; only the prompt changes.
• For the caches: images go in steps of half the cap (8 at the default), counted from the conversation's start (images.left_out): at 17 images the oldest 8 go and the next 7 images change nothing before the prompt, so the head the resident prefix, the state store and the disk tier resume stays the same for several turns (leaving out one image a turn would change the prompt near its start on every turn and miss every cache). Over the row cap the newest message's images are read first, then the earlier ones newest first; the first that would go over and everything older are left out, rounded up to a whole step, so a full queue still holds at most the row cap of patches. The choice is a function of the message list (and the rows of the images it keeps), so the same history always renders the same prompt.
• Only the newest message's own images are never left out: over a cap on their own, the request is the 400 it was, now 17 images in the newest message; the limit is 16 a request, naming the first image over. The refusals of a single message (a broken image, the row cap, the encode budget) are unchanged.
• Each request that leaves images out says so: a log line [images] <request id>: left out the oldest N of M images (the caps: ...) with counts only, the response header X-Tandem-Images-Left-Out: N, extra.images_left_out in the journal's router row, and the counter qse_images_left_out_total (server/METRICS.md).
• The package side, live before this build (packages v7, 2026-10-03 14:24): design-review looks at at most 2 to 3 images a round (one full-page screenshot per width, or a contact sheet stitched from saved shots), never reads an image twice, about 12 in a session, and takes what needs no eyes (overlaps, overflow, contrast, fonts, empty cards) from its DOM audit text; its max_tokens 1600 -> 1800 for the longer text (1,692 tokens on the engine's tokenizer).
• After a compaction the class packages come back (the same session, 14:42 to 14:50): every turn before /compact carried the design packages (conductor_hidden_tokens 6,502); every turn after it carried none, and the model went looking for the Laya endpoint and the design skills on disk. The router's journal said why: opencode 1.18.34 sends its compaction request with its compaction agent's prompt as a system message, which the compaction shape (no system text) ruled out, so the request was routed as design_ui (seven packages, high effort, 2,116 reasoning tokens) and the session never knew it had been compacted; the continue after it was then a new user message, classified chat at 0.998, and chat has no package. The replay was not the cause: a note the history no longer carries is not replayed and the class's block goes in again (tests/test_conductor_router.py had that case). server/conductor/signals.py COMPACTION_SHAPES / compaction_shape: no tools, a system message opening with the agent's fixed instruction, the history as the one user message after it, is the compaction (class summarize, structural, thinking off, no package; the monitor's S2 watch reads the same flag). server/conductor/router.py: the first user message after a compaction keeps the session's class (source: sticky, kept: after_compaction), no classifier asked; the block goes in again at the compacted history's insertion point, the next turns replay it, and the next message a person writes is classified as any other.
• Tests (CPU): tests/test_compaction_turns.py (4: opencode 1.18.34's compaction request is summarize, thinking off, no note, no classifier; after it continue keeps design_ui, the block goes in again (tokens > 0), the tool turn after replays it, the next new message is classified; the same continue without a compaction is classified chat; the shape is exact: tools, a third message, another system text or a flipped order are not a compaction, the old shape still is); the first two fail on 50cc1f5 exactly as live (opencode 1.18.34's compaction request, ('chat', 'qwen')).
• Tests (CPU): tests/test_images.py 8 new (the count rule; the window moves at 17, 25, 33 ... images and nowhere between; 17 images across user, assistant and tool turns: the oldest 8 left out as text, 9 read, the body untouched, a message without a left-out image the same object, the same history the same messages; a broken left-out image is not read; the newest message over the cap is still a 400, and 16 in it with earlier ones are served; the row cap in steps; the checkpoint's chat template (tests/fixtures): 17 placeholders become 9, 8 notes, the text before the first left-out image unchanged; build_prompt with the fake tokenizer: the notes in the prompt, the count path the same length, the body unchanged).

What changes on deploy: the server's code. A conversation over the image caps is served with its oldest images left out; opencode 1.18.34's compaction request is routed as the compaction (summarize, thinking off, no package) and the first user message after a compaction keeps the session's class; every other request is as before.

0.3.20-test, 2026-10-03

68a056c (0.3.19-test, G2's Packages and Skills screen, what :8000 serves) merged with branch tandem-x1 (on 70e1864): the state lines in time order in opencode and Laya playing the build (branch tandem-x1g2). No serve.env key and no ops/ file change; every new field is optional and absent means what 0.3.19-test does: the raw path and every request's bytes are 0.3.19-test's until a profile sets ui.states_channel: inline or a script sends session.

• Why opencode showed every › line at the top of the turn (a user report's screenshots of 09:23 and 10:03, opencode 1.18.34, the coding profile's content channel): read in opencode's binary, its bundled @ai-sdk/openai-compatible (the module createOpenAICompatible is imported from) opens one text part per response at the first content delta (text-start, id txt-0) and ends it only with the stream, so every later content delta, lines and answer, lands in the part the first line opened, above the reasoning; a reasoning delta after a content or tool_calls delta opens a new reasoning part, and the TUI titles a reasoning part with its first line when that is **title** and a blank line. In the 10:03 case (› deciding: chat, low effort, 20 s of thinking, an essay) the whole answer sat above "Thought: 20.1s". In the 10:46 case the follow-up request after two web fetches said › reading 11,426 tokens at ~820 tok/s, about 12 s left twice during its prefill, and the essay was appended to the text part those lines opened, above its 1.1 s thought.
• ui.states_channel: inline (server/conductor/states.py INLINE, server/app.py): on a reasoning_content stream each line is an empty {"tool_calls": []} delta (the provider ends the open reasoning part on it and starts no tool part) and a reasoning delta \n**› text**\n\n, never content: each line opens a reasoning part of its own, titled by it (+ Thought: › thinking 2 min, 4,096 tokens · 1m 31s), in time order, and the answer's text part opens after them. The waiting line of a queued request goes the same way. The strip takes the title form out of the reasoning a client sends back (opencode joins its reasoning parts with nothing between them), so Qwen's prompt ids are the ones without the lines. tags and both (Hermes) get content's stream byte for byte; a non-streamed answer carries the lines at the head of its content. Dashboard: the Profiles screen's "where" has inline.
• Proof in opencode's own parser: tools/opencode_bundle.py takes the provider module (and its imports) out of an opencode binary, tools/opencode_parts.mjs runs a recorded stream through it in node and applies the processor's part rules and the TUI's title. On the box (node 24) with opencode 1.18.32 and with the Mac's 1.18.34: content reproduces both screenshots (one text part on top with every line and the answer), inline gives the lines as titles in order, the thinking under them and the answer last.
• Laya plays the build (server/laya_api.py DRIVER SESSIONS, server/laya_kit/): the intended use, Laya drives the software under test while Qwen writes it. POST /v1/laya/ask takes an optional session: a driver step, rated per session (api.drive_rate_per_s, 10 texts a second; drive_sessions 4 at once; drive_max_texts 36,000), not against the scripts' 240 a minute; no journal row a call, one laya_drive row a session at POST /v1/laya/drive/end {session, summary?} or drive_idle_s (30 s) after its last call: counts, the labels' counts, p, unsure, ms p50/p95/max, the driver's summary (numbers only), the question as a hash, never a state. GET /v1/laya/kit/<file> serves the kit. The server's path is 3.1 ms p50 a step (box CPU, fake model, HTTP); with laya-english's 18.7 to 48.7 ms a text (journal, 2026-10-03) a step is about 25 to 50 ms. The idle rule holds for a playtest (it runs inside a tool call): 14 of the day's 16 laya_api calls met an idle engine, 2 arrived in the last ~100 ms of the request that wrote the tool call and waited for it (148.5 and 187.8 ms).
• The kit: laya_drive.py (stdlib; browser over the DevTools protocol with real keys and clicks in a Chrome it starts or --cdp attaches to, proc over JSON lines, cmd with a state and an action command, a Python Driver; deaths, wins, stuck spots, the step budget, Laya's busy answers with a fallback action; .tandem/drive-log.jsonl; report for Qwen: deaths with place, cause and the decisions before them, stuck spots, the actions, Laya's numbers; serve-log) and laya-drive.js (the same loop inside the page with synthetic keys, for when only the browser tool reaches it).
• The package laya-playtest (texts in ~/tandem-packages/laya-playtest: 294 tokens before its reference, 1,384 with it, on the engine's tokenizer; an inline general library package, so the core goes into the prompt and the recipes are GET /v1/library/laya-playtest): when to use it, the state, the actions, three episodes, the report, and that every interactive build is playtested before "done". laya-testing and test-web get one line pointing at it (335 and 352 tokens). tools/playtest_install.py stores the package, attaches it after laya-testing on every class that lists it (at most 12), and sets a profile's ui.states_channel: inline; dry run by default.
• Tests (CPU): tests/test_state_inline.py (10: the strip, opencode's shape with the lines as titles in order, a tool call with its checks, a line then thinking then the answer, a follow-up after a tool result whose prefill says reading ... left twice (the report's third case, 10:46: the essay drawn above its thought), the replay with the same prompt ids, Hermes byte for byte the content channel, non-streamed, the waiting line, opencode's own parser in node on both channels), tests/test_laya_drive_api.py (9: the card, the session's rate apart from the minute's, sessions at once, the end and its journal row without text, an idle session's row, invalid names, the kit routes, the latency, the status contract), tests/test_laya_drive_kit.py (8: the pieces, proc wins, deaths with place and cause, busy fallbacks, stuck and the step budget, a UI flow over cmd, the report's text, headless Chromium with real keys and the in-page kit through the log sink), tests/test_playtest_install.py (dry run, apply, again, a core that would be cut). Dashboard: tsc, vitest 323, Playwright mock tier conductor and deciders specs 73 passed (desktop 1440, phone 390); dist rebuilt on the box.

What changes on deploy: nothing until the post-deploy stores the package and sets the coding profile's ui.states_channel: inline (a dry run first). The kit routes answer at once.

0.3.19-test, 2026-10-03

70e1864 (0.3.18-test, H1) plus the Packages and Skills screen of the dashboard (branch tandem-g2). The owner, 2026-10-03 09:15: "build the needed GUI for packages/skills in the dashboard". No serve.env key, no ops/ file, no engine file and no conductor document changes; the new routes only read, and the raw path, every chat request's bytes and what the router injects are 0.3.18-test's.

• Four reads, server/conductor/catalog.py (docs/server.md, "The Packages and Skills screen's reads"): GET /v1/conductor/packages/catalog (every package's category -- package, task skill, test skill, card, library skill -- source, path, the tokens of the text that goes in on the engine's tokenizer, of the whole text and of a library skill's reference, its limit: 350 for a library skill and 200 for the library card as tools/library_install.py's house rule, else max_tokens, and whether it is over; the classes of the router and of each profile that list it; INDEX.md's fit; each profile's library switch), GET /v1/conductor/packages/item/{name} (frontmatter, body, reference, the injected text, their tokens, a library skill's six BM25 neighbours), POST /v1/conductor/packages/tokens (1 to 20 texts, the editor's counter; a read) and GET /v1/conductor/packages/usage (the journal's 7 days per package: injections, shadow notes and library picks in 24 h and 7 d; one package's newest rows; the library's newest 50 retrieval rows). Contract files packages-catalog, packages-item, packages-tokens, packages-usage. server/conductor/packages.py names its cut note CUT_NOTE (the same text).
• Dashboard, Conductor > Packages, rebuilt: one list of every package with kind, source, switch, general flag, tokens against the limit, classes and its use in 24 h and 7 d; a search (the library through GET /v1/library/search, ranked by the server; the rest by name, title, description and source) and filters (kind, class, enabled, over the limit). A package's detail: the text that goes in rendered or raw, the frontmatter, the reference, the path; the Edit pane (title, description, max_tokens; an inline package's text in a Markdown editor with write, preview and split views, the engine's token count a moment after typing and the kind's limit warning; a file on the box read-only, or "Copy to inline" to override it); the Classes pane (attach and detach on the router's classes, general packages first, at most 12, a library skill's INDEX.md fit instead); the Usage pane (counts, the newest rows with their sessions, a library skill's neighbours). Every save is checked first (POST .../check), its diff shown, then a new version with a note (PUT); the packages' and the router's versions with restore; the admin token asked for when the page may not write. The Library tab: each profile's library switch (off, shadow, on; checked, its diff in the confirmation, a new profile version) beside what the retrieval picked in the journal (candidates and scores, the pick, shadow or on), so the shadow can be judged before a profile goes on. Phone layout at 390 px: the rows stack, the detail takes the screen, nothing scrolls sideways. shared.ts gives the versions panel its own function (the same markup).
• Tests (CPU): tests/test_packages_catalog.py (7: the catalog's categories, tokens, limits, classes, fits and switches; the estimate; one package and its neighbours; the tokens and their limits; the usage's counts, rows and library rows; the routes are reads and a PUT still needs the token; nothing moves: the config directory's files, the versions and every injected text the same after every route). Dashboard: vitest tests/packages.test.ts (15) and the contract test's mock responses for the four routes (3); Playwright e2e/packages.spec.ts (11, desktop 1440 and phone 390: list, filters, search, detail, edit with check, diff, save and the token, a new package, attach and detach, a version restore, copy to inline, the library tab), e2e/library.spec.ts follows the new list.

What changes on deploy: the server's code and the dashboard's dist; no document is stored and nothing served changes.

0.3.18-test, 2026-10-03

03910b1 (0.3.17-test, R1's package library over 0.3.16-test 2a2a0c9) plus the disk tier's admission for a new session in another folder, and the smoke set's heartbeat check (branch tandem-h1). No serve.env key, no ops/ file and no conductor document changes; the raw path and every request's bytes are 0.3.17-test's, and a restore is the tier's (bit for bit a cold prefill's).

• Why a new opencode session started cold with the disk tier on (2026-10-03 00:09, the eval's minicalc session: 25,616 prompt tokens, cached 0, TTFT 35.3 s). The heads on disk of two eval sessions in two folders were equal for 13,954 tokens: the tool block (to about 12,000) and opencode's fixed prompt, up to the <env> block's working directory; the instruction files, the skills list and the user's message follow it. The first session's head anchors were at 20,480 to 23,552, past that point, and a line is admitted at its second cold read, which was the minicalc session's (the eval empties the caches in RAM before each run); it wrote the anchor at 13,312. A head with two MCP tools more parted at 9,468.
• A branch line (engine/disk_cache.py plan): a line counted admit_after times whose count falls at the next line gets an anchor beside the longest repeated line, the 4 longest (BRANCH_MAX), only up to where the conversation's first answer starts (server/diskcache.py request_tag: first_answer, the third <|im_start|>).
• The counts outlive a new fingerprint on the same grid (index.json keeps the grid): a deploy that changes the engine's code swept the counts with the files, and the first two sessions after it read the shared head cold. Sessions, hits and stats stay the files'; the dashboard's clear still drops the counts.
• tools/client_smoke.py's prefill heartbeat check: S1's progress lines open the stream while the prefill runs, so the heartbeat's comments came after an event and the check failed on TandemLLM (0.3.15-test's verification, 27/28). read_heartbeat accepts a comment before the model's first token, after events that carry only step lines (›), and refuses one after a token or inside an event (between its data line and its blank line); every data line must still be JSON and the stream end in [DONE].
• Tests (random model, CPU): tests/test_disk_cache.py (the counts across a new fingerprint and not across another grid, the files swept; two sessions in two folders, a deploy, a session in a folder seen before writes the branch line, a new folder resumes from the disk there, bit for bit a cold prefill; branch lines only in the head and the 4 longest), tests/test_disk_cache_server.py (first_answer), tests/test_client_smoke.py (the heartbeat reading: comments before the first event, after step lines, refused after a token or a tool call, inside an event; an intact stream). The three disk tests fail on 2a2a0c9's and 03910b1's code.

What changes on deploy: the engine's code changed, so the disk tier's files are swept at the start and its counts kept. Expected: the first opencode session after the deploy writes the anchor at the line below the <env> block (13,312 for the eval's config) and the next session in another folder reads it instead of prefilling it (about 18 s of 35 s at 25,616 tokens).

0.3.17-test, 2026-10-03

2a2a0c9 (0.3.16-test, L1: Laya for the model's scripts) plus the package library (branch tandem-r1lib). The owner, 2026-10-03: "multiple packages about many things and subjects so the model can search and use them later"; 44 skills were written (~/tandem-packages/library). No serve.env key, no ops/ file; every new field is optional and absent means what 0.3.16-test does.

• The index (server/conductor/libsearch.py): BM25 over every enabled package with library: true (new optional packages field): name x3, description x2 and the text before ## Reference; INDEX.md's class fit as a filter; rebuilt when the packages version, a SKILL.md or INDEX.md moves (looked at once a second at most). 44 skills: build ~2 ms, a 900-character query ~37 us (box CPU). A library package's injected text is its part before ## Reference; the reference only on a read.
• Retrieval by the router under a new profile switch library: off|shadow|on (absent: off; nothing runs and the index is never built). On a new user message, and on a tool turn whose J1 outcome is in library.outcomes (low_quality, stuck), the digest is searched, the top 4 go to the existing package question (packages.fit, the decider or Qwen, or none), up to max_picks (1). On: the skill is a note at the prompt's tail, replayed at its position; a live skill is not offered again; an outcome's pick counts as an act. The tandem-library card (191 tokens: the library exists, search and read it with curl) stays live on the coding classes with tools. Shadow: extra.library, shadow.library and shadow.library_card only. A plan decided under the engine lock asks the decider stages only (locked when none). The router document's optional library section holds the knobs.
• Found, not changed: the class's own package question in the in-context stage (under the engine lock) asks Qwen through the server's own port and waits on the lock it holds (journal 2026-10-02 14:22: qwen_timeout after 8,009 ms, classifier 33.8 s); the in-context stage is off in router v9.
• GET /v1/library, /v1/library/search?q=&k=&class= ([{name, description, score}]), /v1/library/{name} ({name, description, classes, body, reference}), server/library_api.py: open as the chat API and L1's Laya route, CORS, OpenAI-shaped errors (422, 404, 429 over 240 calls a minute), one journal row a call (kind: library_api, the query as a hash). Contract library, library-search, library-skill.
• tools/library_install.py: the files to ~/.qwen38-spark-engine/conductor/skills/library/ and the card, the packages (skill:<name>, library: true, max_tokens 500, the library directory in skill_dirs), a profile's switch; dry run by default, --measure on the engine's tokenizer, nothing trimmed. Measured on the Qwen3.8 tokenizer: bodies 305 to 395 tokens (total 15,198), 16 of 44 over the 350 the house rule names, listed in the install's report; references 241 to 719 (total 22,612).
• Dashboard: the Packages screen's Library panel (the server's own search, scores, classes, the editor), the class packages without the library (Routing, Profiles), a profile's library switch; mock routes.
• Tests (CPU): tests/test_libsearch.py (the build, the search, the class filter, rebuilt on change, the injected text, time, the real library: seven task sentences find the skill a person would pick in the top 3), tests/test_library_router.py (off identical to no library, shadow serves off's prompt and journals the pick, on injects and replays at the same position, a live skill, a compaction, a stuck outcome, none, two picks, no match, the locked path, the profile in shadow, the test box, the schema), tests/test_library_api.py (open, search, read, list, the contract files, limits and errors, the journal rows, a file change), tests/test_library_install.py (dry run, apply, a second run, a text the server would cut).

What changes on deploy: nothing until tools/library_install.py --apply registers the library and a profile sets library; the plan is library: shadow on the coding profile first.

0.3.16-test, 2026-10-03

3cf9093 (0.3.15-test, what :8000 serves since 23:16 on 2026-10-02) plus Laya for the model's own scripts and the test judge on its own role (branch tandem-l1). The owner, 2026-10-02 18:45 and 2026-10-03 00:20: the model must reach Laya through our provider, from the scripts it writes, and must use it to test any software. No serve.env key and no ops/ file change; the raw path and every chat request's bytes are 0.3.15-test's.

• POST /v1/laya/ask and GET /v1/laya (server/laya_api.py): a typed verdict on a short text, open like the chat API (no key at the engine, LAN-only behind the proxy). {question, choices | scale | neither, text | texts, threshold?, model?, deadline_ms?} answers {label, p, sure, probs, model, ms, chars, read_chars, clipped} (a scale adds level and score, a yes-no question yes; texts answers one a text). Laya reads its window of a long text, the head and the tail, cut by code (1,120 characters for laya-english). Idle window only, as /v1/systemone: a caller that meets a running completion waits up to its deadline (15 s) and gets 503 busy. OpenAI-shaped errors (422 before any model runs, 429 over rate_per_min, 503, 404 off). Access-Control-Allow-Origin: * and an OPTIONS preflight, so a test inside the page (the browser tool's evaluate script) can fetch it; every other path's OPTIONS is the 501 it was. One journal row a call (kind: laya_api), the question as a hash, never a text. The deciders document's api section (on, 240 texts a minute, 16 texts a call, 100,000 characters a text, 15 s, sure at 0.8) and questions.laya_api; the Deciders screen's "Laya for scripts" panel edits it and shows the calls; GET /v1/conductor/deciders/status reports it as api. Contract: laya-ask.schema.json, laya-ask-texts.schema.json.
• The test judge on its own role. J1's judge (Laya on the narrowed cases, shadow) asked through the router's decider role; with routing Qwen's since router v7 it wrote no row in 0.3.15's verification (the served coding profile also had outcomes.judge absent, so off). A built-in role tester (kind decider, endpoint self, model auto, mode on, window idle; a roles document stored before it gets it) is the judge's now, the deciders document's questions.judge may name the model, and the digest is cut to what the model reads, head and tail, before it is asked (the decider client kept the front: a log's tail was lost). The profile's outcomes.judge: shadow switches the questions, as before.
• A class may list 12 packages (was 8): the coding classes list two general packages more.
• Packages (texts on the Mac in ~/tandem-packages, installed by ~/l1-install.py): laya-testing (296 tokens, general on code_edit, debug, design_ui, three_d, drawing: what Laya is, the call, and that the model must ask it about its evidence after every run and before "done"), laya-scripts (228 tokens of core and a reference of 937 with the helper .tandem/laya.py and recipes for logs, CLIs, APIs, pages and games, data and regressions; general on code_edit and debug), one Laya example in each of test-web, test-cli, test-api, test-data, test-lib, and the testing package and the harness card no longer say that the runtime judges every run (it did not: the judge is in shadow).
• Tests (CPU): tests/test_laya_api.py (12: the card, a choice open through the proxy with and without a bearer, CORS and the preflight, a yes-no, a scale, texts with one forward each, a long text read by its head and tail, 16 invalid requests refused before any model, a busy engine, the journal without text and the status counts, the switch and the rate, off is the release), tests/test_judge.py (the tester role: the router's decider role and stage off and the judge still asks; the built-in role for an old roles document; the named model; the digest cut to the model's window). Dashboard: vitest (the status, the seeds, the two response contracts), Playwright (the panel saves mode and rate as a new version).

What changes on deploy: nothing in serve.env. The route is on with the defaults (no deciders document is stored). The post-deploy installs the packages (a dry run first), attaches them to the coding classes, sets outcomes.judge: shadow in the coding and overnight profiles, and runs the live check.

0.3.15-test, 2026-10-03

7954241 (0.3.14-test, what :8000 serves since 22:04 on 2026-10-02) plus the fixes for the two failures of its post-deploy live check (branch tandem-d2). No serve.env key, no ops/ file and no conductor document changes; the release's text is 0.3.14-test's.

• The end anchor never engaged live (2026-10-02 22:07, ~/c314-live.py (a): raw turn 1 138 -> 1,413 tokens, its follow-up of 1,869 tokens cache_n 0, source none, TTFT 3,743 ms). The client's copy was not the cause: in the box's suffix store the follow-up held the generated ids token for token through <|im_end|>. The router was: the raw model's name is an alias that plans as TandemLLM, and turn 1 got a 66-token note (effort low) inside its user message at token 65. A conversation with no session id is keyed by a hash, and the hash was of its first two messages: the user's alone on turn 1, the user's and the first answer on every turn after. The follow-up was another session (h:c1c5... then h:4e49... in the journal), turn 1's note was not replayed, and the prompt parted from the resident rows at token 65 -- below the end anchor, below any grid line. The key is now the conversation's head, its messages up to and including the first user message (server/conductor/signals.py first_hash); with a system message that is the same two messages as before. Every system-less chat (the raw name, Hermes, Open WebUI) lost its notes and its resident prefix at turn 2 the same way.
• The hidden close's phrase and the template's newline (the same check, (a2): TandemLLM with a client budget of 400, the phrase hidden at 413 reasoning tokens). The replay puts the phrase back where the rows hold it, but the client's copy is re-rendered by the template, reasoning|trim + "\n</think>", and the phrase ends in its own newline: the copy held one \n more than the rows right after it (suffix store: the follow-up parted at the token after Mod 7, \n before </think>), so the replay found no match and the turn re-read the whole answer. ResidentPrefix.replay aligns runs of whitespace-only tokens next to a hidden span (glue, the tokenizer's single tokens for newlines, spaces and a tab, server/app.py glue_ids, set at load); every other token must be the rows', and what is served up to the end anchor is the rows' tokens. usage.prompt_tokens is the client's count, net of the alignment. This is the rrr2 case the end anchor was built for (the D2 backstop closes with the same phrase).
• Not fixed, documented (docs/caches.md): (a2)'s turn 1 (459 tokens) came right after the 1,932-token raw conversation and was a guest of it (shorter than a quarter of it), and a guest keeps no end anchor -- its rows are written over the resident prompt's stashed ones, which the next request puts back. A short conversation that starts while a longer one is resident re-reads its first answer at turn 2, as every guest re-reads its whole prompt. ~/c315-live.py gives (a2) a 9,000-token system text, over --resident-guest-max.
• The disk restore's text "mismatch" (2026-10-02 22:08, tools/disk_cache_live.py: 15,360 tokens restored, 1.79 GB in 334 ms, same_text false) was the check's: it compared the raw stream, and the stream held TandemLLM's step lines (› reading 16,427 tokens at ~800 tok/s, about 18 s left twice on the cold run, › waiting: another request is running, position 1 on the disk run) and the tool calls' ids, which the server draws per request. The model's text and the calls' names and arguments were the same: "Let's start by exploring the project to understand what we're dealing with." and tool_00/tool_01 with {"path": "."}, both runs. The check now compares the text without the step lines (states.py strip_text) and each call's name and arguments, and prints the server's TTFT (23,106 ms cold, 2,681 ms from the disk on 22:08; the client's first piece is a step line). On the GPU (tools/disk_restore_check.py, in a hold, the served flags and drafters, a directory of its own) the restore is bit for bit a cold prefill's: a 10,701-token opencode-shaped prompt, the head written behind the request (10,240 tokens, 4 anchors, 10 chunks, 1.71 GB), read back into clobbered buffers in 226 ms: the recurrent states and convolution tails at line 10,240 equal the cold prefill's own at that line, the prefill resumed from the disk equals the cold prefill in the last row's logits, the states, the tails and all 6 position-indexed buffers (target K and V, both drafters' rows), and the served loop writes the same 48 greedy tokens from the disk as cold. In a second process (a restart, so a fresh Triton autotune) the cold prefill is the first process's bit for bit (logits, states, tails, rows, the state at each of its 10 lines), the first process's files restore to it, and the text from the disk is the first process's cold text (hold of 2026-10-02 22:32, ~/d2-diag/{write,read}.json).
• tools/end_anchor_identity.py --on-base: against a base that has the end anchor (7954241), the candidate's turns with it on must equal the base's on as well.
• Tests (random model and the real chat template, CPU): tests/test_conductor_router.py (a chat with no system message and no session id keeps its session at turn 2 and replays turn 1's note; turn 2's prompt begins with turn 1's served prompt), tests/test_end_anchor.py (the template's newline after the phrase aligned: the copy resumes at the end, served with the rows' tokens, and without glue it parts there as in 0.3.14; the rows' whitespace before the phrase that the copy's trim dropped; anything but whitespace next to the phrase serves the copy as sent; glue_ids), tests/test_disk_cache_live.py (step lines and call ids are not the text; other arguments or another text still fail).

What changes on deploy: nothing in serve.env or the conductor's documents. System-less chats keep their router session from turn 2 on (their class sticks, their notes replay). The post-deploy runs ~/c315-live.py ((a) and (a2) as gates) and tools/disk_cache_live.py, and turns the disk tier on only when the latter passes with a disk hit and the same text.

0.3.14-test, 2026-10-02

4f47239 (0.3.13-test: K1 c3ade36, C1 f11c0c6 the disk tier, J1 6df4abf snapshots, outcomes, cards and the judge; the hold of 2026-10-02 at 20:51 deploys it) plus branch tandem-s1 ebeb071 (on 4764ae2: a client's side requests served as what they are, the state lines outside the thinking, progress numbers in them), merged as branch tandem-m2, and one engine change: the resident prefix's end anchor, so the turn after a long generation forwards its new messages instead of re-reading the generation. serve.env gets one key (END_ANCHOR=on); ops/start.sh passes it.

• Side requests (structural.side_requests in the router document, on; off serves them as turns, as before). opencode's title and summary calls and Hermes' title call are known by their shape: no tools, and the client's fixed instruction opening the system message (opencode 1.18.34's title and summary agents, Hermes 0.21.0's title_generator; structural.side_shapes replaces the list). Such a request asks no classifier, gets no package, rules or note, leaves the session's class, turns, notes and size alone, thinks only when the client asked for a number of reasoning tokens, and writes at most its shape's cap (64 tokens for a title, 256 for opencode's summary). Journal: kind: side, in the decisions feed beside the turns and out of the Overview's and the sessions' figures. In the queue a side request of at most 8,192 prompt tokens goes ahead of every waiting turn (a few seconds of engine); a longer one waits until no turn does. On 2026-10-02 at 18:08 opencode's title request was classed code_edit at xhigh, wrote 5,567 tokens and held the session's turn 54 s.
• The state lines outside the thinking (ui.states_channel in a profile: reasoning, as before and for every profile stored before it; content, the seed profiles' value). In content every line is a content delta of its own: before the reasoning, between its parts, after it; in tags a line said inside the open block waits for </think>. The strip already took the lines out of an assistant's content, so the prompt is unchanged; a non-streamed answer carries them at the head of its content.
• Progress numbers (ui.progress, on with the lines; prefill_every_s 10, thinking_every_s 120, thinking_every_tokens 4,096): waiting: opencode's title request runs first, position 1 (another request is running) while a request waits, reading 35,189 tokens at ~750 tok/s, about 47 s left every 10 s of a long prefill, thinking 4 min, 9,800 tokens every 2 minutes or 4,096 tokens of thinking; templates, never the narrator's. A request that waits opens its stream at its first waiting line; a refusal after that (queue timeout, prompt too long, memory) ends the stream with an error (its code) and [DONE], and the ledger keeps the status.
• The end anchor (--end-anchor on|off, END_ANCHOR in serve.env, on; engine/cache.py ResidentPrefix.keep_end, server/app.py _remember). The re-read after a long thinking (rrr2: 33,172 tokens, about 44 s) was not the hidden close phrase: every turn of that session, and of the 18:08 one, resumed at the previous prompt's last 1,024 line whether a phrase was hidden or not, because the resident prefix anchors prefill chunks only and a long session's snapshot is over the store's entry cap (S1's tests/test_reread_after_think.py). At the end of every generation the resident prefix now keeps one anchor more: the recurrent states and convolution tails at the last position the engine forwarded (147 MiB, two buffers reused turn after turn), the rows in place valid up to there. A generation that stopped inside an accepted block has its pending commit cut back to the committed tokens first (trim_pending); one whose state already absorbed them keeps none. The next request that starts with every one of those tokens resumes there and forwards only what is new; it is taken when it starts later than the anchors in place, the pin and the park, and a store entry or the disk only when they start later still. The journal's cache_source is end (the live and requests contracts list it). A resume there is the session cache's kind of resume, held to the greedy gate, and so is every anchor a prefill from there takes: the resident prefix's taint is the lowest such row; the store takes no prefix snapshot of such a prompt, the disk tier writes nothing of it and reads over those rows, and a grid anchor at or below it, or a cold prefill, is bit for bit a cold grid prefill's again. The pin and the park carry their rows' line. The end anchor goes with its rows (a prompt that parts inside the generation resumes at the prompt's last line, as before). Memory: asked of the policy once at the new end level, given back after the body anchors and before the guest stash. Not for a request with images.
• The hidden tokens put back. What the session monitor hid in an answer (a hidden close's phrase, a hidden turn: _force_close records each span) and SP4's note at the prompt's tail are in the rows, not in the client's copy, which then parts from the end anchor at the first of them. Before the prefill (after the router's notes, before SP4; and for the in-context stage's prompt) the server puts them back where they sit when the prompt then starts with every token up to the end anchor (ResidentPrefix.replay), else serves the prompt as sent. They stay hidden: not in prompt_tokens, the suffix store or the response cache; the journal row has extra.end_replayed; the next end anchor carries them on.
• Whether the client's copy matches: in the box's suffix store (2026-10-02, 4,607 turns that followed another, token ids only) the next prompt held the engine's visible output token for token to its end in 3,844 (83 %); the others parted late (91 tokens before the end at the median) and resume at the prompt's line, as before.
• C1's disk tier, two bugs seen live on 0.3.13-test (2026-10-02 21:09, tools/disk_cache_live.py FAIL: no disk hit). (1) The writer's CUDA calls (its side-stream copy and synchronize, the pinned staging buffers) ran while the drafter captured its CUDA graph, the first capture after the restart: the copy failed (cudaErrorStreamCaptureUnsupported) after 1 of the head's 15 chunks, and the capture with it (an INTERNAL ASSERT in the caching allocator), so the request ended in an error. A process-wide lock (engine/capture_lock.py) is held by every capture (engine/drafters/draft_graph.py, engine/verify_graph.py) and by the writer around its CUDA calls, and by the probe writer's event wait. (2) The head's 4 anchors were on disk without their chunks (complete: false), so the next session found nothing to read and prefilled its 17,074 tokens cold (22.5 s server-side, with the RAM caches just emptied); and plan counted those anchors as written, so no later request would have written the missing chunks. An anchor now counts only with every chunk below it, and such a head is admitted again. tools/disk_cache_live.py's gates: the head must be complete with no writer error, and both compared requests must finish without an error (on 21:09 same_text passed on two errored requests). Tests: tests/test_capture_lock.py (every capture site under the lock; the writer copies nothing while it is held; a head whose chunks failed is admitted again and then read bit for bit; both disk tests fail on 4f47239's tier), tests/test_disk_cache_live.py (the failed run is a FAIL); tools/capture_race_check.py: the real writer against graph captures on the GPU, with the lock and without (the control), a hold step.
• S1 with C1: a client's side request is read by the disk tier and admits nothing (request_tag's side); it already wrote no J1 snapshot, outcome or card (it returns from the plan before them).
• Tests (random model, CPU): tests/test_end_anchor.py (the served loop with no drafter and wrong and right chains and trees: the turn after a 150-token generation resumes at its end and writes the 24 greedy tokens the same turn writes from the prompt's line; a pending commit cut back is the shorter commit's state bit for bit; the replay, its carry-over, SP4's span, the in-context stage; taint, the store, the disk, the pin, the park, a guest, memory, the length router's released arm inherited; through the handler with usage the client's); tools/end_anchor_identity.py: the end anchor off is the base tree's resident prefix bit for bit (tokens, resume points, state, rows, anchors, pin, park), on differs.
• tools/stream_identity.py: two trees' streams on the fake engine, byte for byte, with S1's switches off (or at their defaults, where only the title request may differ).

What changes on deploy: the disk tier stays as the settings say (off since the 21:09 FAIL, settings v1); the post-deploy check turns it on, runs tools/disk_cache_live.py and leaves it on only when the head is read from the disk with the same text. Opencode's and Hermes' title requests are served as side requests at once (the stored router has no side_requests, so it is on). The coding profile on the box (v8, ui.states: on, no channel) keeps its lines in the reasoning and gains the progress lines there (on by default with the lines); its stream now opens while it waits for the engine. Moving coding's lines into the content is a profile version (ui.states_channel: content, Profiles › What the user sees). The end anchor is on: a turn that sends back the whole previous answer resumes at its end (cached_tokens = the previous prompt plus its generation), 147 MiB more under the 70 GiB ceiling; END_ANCHOR=off and a restart is 0.3.13-test's resident prefix. A rollback to 0.3.13-test (4f47239) ignores END_ANCHOR, and like one to 0.3.12-test refuses a router version saved with side_requests or side_shapes and a profile version saved with states_channel or progress, and serves the newest older version.

0.3.13-test, 2026-10-02

c3ade36 (0.3.12-test 4764ae2 plus K1, branch tandem-k1: the claim check E2, general packages, the eval's effort to PASS; what :8000 serves since the hold of 2026-10-02 19:12, its entries are the last part below) plus two branches merged onto it as branch tandem-m1: C1 (branch tandem-c1 f11c0c6), the disk tier the owner asked for on 2026-10-02 -- the ~24-30k-token head every opencode session starts with, the prefixes prefilled more than once and the packages in them, kept on the NVMe under a budget set in the dashboard; and J1 (branch tandem-j1 6df4abf), the turn snapshots, the Tier 0 turn outcomes, the client tool card and the harness card, the evaluation points, Laya as the test judge in shadow and retrieval on low quality (the plan "Laya as the test judge, state snapshots, turn outcomes and the client protocol", steps 1 to 4 and 6 to 8; step 5 is package texts, step 9 not built). The merge keeps both: the router's plan carries C1's spans beside J1's snapshot, outcome and extra notes, and the disk tier's spans cover J1's notes too (an entry whose notes are an outcome's note or a card is labelled a prefix on the Disk cache tab, not a package: a card's parts are of kind card). The disk tier is off in the code; serve.env gets one key (QSE_DISK_CACHE), which the deploy sets to ~/.qwen38-spark-engine/cache/disk and ops/start.sh passes to the :8000 service only (a row, gate or soak server never has the tier). With the tier off, every prefill runs the code path of c3ade36 (the prefill takes disk=None); with it on, a request that starts from the disk computes the same bytes as a cold prefill on the grid, and the served text is the same (tests). Every J1 part is a switch in a profile's new outcomes section, off when absent; the router document's optional outcomes section holds the settings over the defaults. The engine's decode path is c3ade36's; the served loop gains one call before the prefill (Qwen asked in place on a narrowed case, off unless the router document's outcomes.evaluate.ask.mode is shadow) and the monitor's claim check one optional text.

• The disk tier (engine/disk_cache.py). Chunk files hold 1,024 tokens of rows of every position-indexed buffer (the attention KV and the drafters'), keyed by the prefix hash, with the token ids inside; anchor files hold the 48 recurrent states and their convolution tails at one line, with the chain of chunk hashes below it. Written with O_DIRECT, to a temp file, fsynced and renamed; crc32 per section; the files are the index. What is written: the longest line a prompt has had prefilled cold admit_after (2) times, and the first long request of a session at its last 4 lines below where its user message starts; the chunks below them not on disk yet. An anchor's state is copied to a staging slot as the prefill passes its line; the rows are written behind the request from the engine's buffers until the next prefill fences them (a write that cannot finish in that window is dropped and journaled). The lookup comes after the resident prefix, the pin, the park and the state store, takes the disk only where it starts later and its read costs less than the prefill it saves, reads only the chunks above the rows in place, compares every stored token id with the prompt's before a byte reaches the engine, and makes the restored prompt the resident one with the disk anchor as its anchor. A file that fails its check goes with every longer chunk of its chain, and the request resumes from the rows in place or starts cold. Eviction least valuable first (hits, tokens saved per byte held, halved per day unused), a TTL (7 days), and never under 50 GB free on the NVMe. A file of another fingerprint (weights, engine code, arithmetic flags, grid, library versions) is swept at the start. The staging buffer (2 anchor and 2 chunk slots, 0.5 GB on the served stack) is its host footprint, allocated at the first use and counted in the engine's declared peak until then.
• Settings: settings.memory.disk_cache (mode, budget_gb 50, ttl_days 7, admit_after 2, min_free_gb 50, which the schema holds at 50 or more), versioned like the transcript settings, applied at the next request. The Memory screen's new Disk cache tab: the bytes against the budget, the NVMe's free space, the hit rate, the tokens restored and the last restore's time and rate, the most-hit prefixes (token counts; a preview for opencode heads and packages only), the last events, the settings, a clear button; phone layout as cards. Routes: GET /v1/conductor/memory/disk, POST /v1/conductor/memory/disk/clear, POST /v1/cache/clear?tier=disk|all; /v1/cache/stats gains disk_cache (counts, no previews). The journal gains kind memory, task disk_cache rows (hit, write, evict, drop, restore_failed, clear, error); /metrics gains qse_disk_cache_*. A request restored from the disk has cache_source disk in the ledger and the live view (contract: additive).
• tools/disk_cache_bench.py (CPU only, a temporary directory): on the box, O_DIRECT writes at 4.3 GB/s, reads at 7.35 GB/s, crc32 at 5.6 GB/s. A 30,720-token head is 3.43 GB: 0.47 s at the read rate, about 0.6 s with the checksum, against about 44 s of cold prefill (arithmetic; the board's restore is measured after the deploy). tools/disk_cache_live.py: the post-deploy check (a warm-up session writes the head; a fresh opencode-shaped session's first request reads it; TTFT against a cold run of the same prompt; the same text).
• J1: the turn snapshot (outcomes.snapshots: on, server/conductor/outcomes.py). At the start of every request whose newest message is a tool result the router builds, from the body alone, what the session did since the last one: the calls (8, results clipped to 1,200 characters), the files written (path, size, a hash), the last write and run, each screenshot's hash and size, the console's level counts and error lines, the checker lines and the session's best per gate, exit codes, a test runner's counts, the tandem_state dump and its diff, a .tandem/*.log tail. The transcript writer stores it as snapshots/turn-<n>.json beside block n (0600, at most 32 KB) with the images under shots/ (the newest 5 a session), one snapshot: line in the block's runtime section and counts.snapshots in meta.json. 0.023 ms a request (median over the two Mario sessions, box CPU). Pure observation: the plan and the prompt are the same.
• Tier 0 outcomes (outcomes.mode: shadow | act). One outcome a tool-result turn, the first label of the router document's outcomes.order whose facts matched (no_visible_change, regression, error, low_quality, surprise, stuck, tool_misuse, test_faked, environment, lost_context), progress, or none. Every fact is structural: an equal hash after a write, a worse number, an exit code, a runner's counts, a console level, an OS error string, a call the client refused, a script that assigns to a member of an object it did not declare. Journaled as kind: outcome with its evidence; the route row has extra.outcome; tools/tandem_eval.py counts them per session (efforts.outcomes, schema additive). Act: no visible change and regression set the turn to xhigh, error to high (never below the class's), held for raise_turns (2) tool turns or until a progress; the outcomes in acting (the three) add their note once per signature (the E-a note with the snapshot's digest, at most 2,100 characters), replayed at the same token position like every router note, counted as an act under the profile's caps (the monitor adopts it); at the cap the effort still changes and the note is dropped.
• The client tool card and the harness card (outcomes.cards). For a coding class with tools: the request's own tools (names, required arguments starred, which take a path) with the client kind's pitfalls package (tools-opencode, tools-hermes), and the tandem-harness package, each kept live in the session's notes (in on the first turn and again after a compaction dropped them). A tool-misuse outcome brings the tool card back as its note.
• Evaluation (outcomes.evaluate, step 6). E-c: stuck and low quality go to xhigh with the re-plan note and the digest. E-b: the claim check's note carries the turn's digest (act; shadow journals its size). Qwen in place on a narrowed case (a run's output with no exit code and no counts), through the in-context stage's forward before the turn's prefill, journaled beside the twin, shadow only and off by default (outcomes.evaluate.ask.mode).
• Laya as the test judge (outcomes.judge: shadow, server/conductor/judge.py). On the narrowed cases (failure: a run's output without an exit code or counts; build: a .tandem log tail) Laya is asked after the request on the judge's own thread, through the decider role, with a 100 ms deadline; each answer, timeout or failure is a kind: outcome row (task judge) beside the Tier 0 twin. It acts on nothing. tools/judge_report.py gives the confident precision and coverage per question and the gate (0.95 and 0.3).
• Retrieval (outcomes.retrieve, step 8): a low-quality or stuck outcome asks the router's package question over retrieve.packages (the kind skills) and the package that fits is the note, once per signature.
• The secret-pattern refusal covers J1's text as it covers the transcript blocks: every string of a snapshot file, every outcome's evidence line and every digest that goes into a note.
• The Memory screen: a session's line counts its snapshots and each block's runtime section names its snapshot with the turn's outcome (dist rebuilt; the memory schemas' counts allow snapshots).
• K1 (live as c3ade36): the claim check, E2 (server/conductor/monitor/claimcheck.py, ops/supervisor.yaml version 5, shadow). A coding turn (the router's class in code_edit, debug, design_ui, three_d, drawing) of a session with a write behind it that ends on a non-empty answer with no tool call: Qwen itself answers, in place under the request's engine lock, thinking off, one letter each, whether the message claims the work is done (Q1) and what the tool evidence since the last write says (Q2: supported, no_run_after_last_write, errors_in_evidence, surprise_unexplained, test_faked). An answer other than supported continues the turn hidden with its note; everything else takes the end token as drawn (fail-open). Once a request, two notes a session, under the caps; journaled at tier 1 with both answers, the structural twin and the calls since the write, and an outcome label (fixed, ran_no_fix, no_run) later. The profiles' E2 (shadow in coding and overnight) sets its mode, claim_at and admit_at; E2 needs no role any more. The two Mario sessions of 2026-10-02 are the replay fixture: it fires on both final messages.
• K1 (live as c3ade36): general packages: a package with general: true in the packages document goes in whole on every turn of a class that lists it, before the task skill the router picks among the rest (so three_d and drawing can carry the coding package and their recipe).
• K1 (live as c3ade36): tools/tandem_eval.py: tool calls and turns to PASS beside the minutes to PASS, per run and per suite and arm; every comparison and both report tables read the records' minutes_to_pass, never the wall minutes; the pass rule decided on 2026-10-02: a run of a suite with a final check passes by its verdict, the stage gates are progress. The report schema's new fields are optional.

What changes on deploy: what TandemLLM and raw answer does not change. C1: the first long request of each opencode session and every prefix prefilled twice are written to ~/.qwen38-spark-engine/cache/disk (0700) behind the request, up to 50 GB; the next session's first request starts from the disk. Turning it off is mode: off on the Disk cache tab (no restart; the files stay) or QSE_DISK_CACHE=off and a restart. J1: the deploy stores profile versions of coding and overnight with an outcomes section (snapshots on, the outcomes in shadow, the judge off), as j1-install.py does; no stored router has an outcomes section and the package texts (the tool and harness cards, the kind skills) are installed after the deploy by J1's installer (a packages version and a router version, dry run first), so nothing is injected until then; the Memory screen shows the snapshot count and line where a session has them. K1 is live already (its monitor document version 2 with E2 in shadow stays as stored). A rollback to c3ade36 leaves the disk directory and the snapshot files on the disk (it neither reads nor deletes them), refuses a settings version saved with memory.disk_cache (it serves the version before) and restores the coding and overnight profiles to the versions current before the deploy.

0.3.12-test, 2026-10-02

0.3.11-test d539719 (tandem-f2) plus ENG-181 (branch tandem-p2, built on 62d9f5a and merged onto d539719; epic VIS-E7, R2 of the roles and harmony design): snapshots of Qwen's own rows at the router's three decision points, the tool that trains the first probes on them, and the router's probe stage, every part off by default. The engine's decode path and ops/ are 0.3.11-test's; the loop gains a probe argument that only a request with the snapshots on passes, and the block drafters a read-only tapped(). serve.env gets two keys (QWEN38_PROBE_SNAPSHOTS=off, QWEN38_PROBE_DIR). With the snapshots off, TandemLLM's output is 0.3.11's and raw is byte-identical; with them on, the output is the same (tests on every verify path: tokens, context, KV and recurrent state equal with and without a capture).

• Probe snapshots (server/probe.py, QWEN38_PROBE_SNAPSHOTS=on; a design decision of 2026-10-01: box only, 30 days, tool requests only). For a TandemLLM request with tools (raw never, chats without tools never), one row per tapped layer (the five layers the block drafters read, 5, 19, 33, 47, 61) and the post-norm row, fp16, at the last message's last token (class and effort: the row the router's note and the thinking splice do not move, so a probe can read it before the router decides), every 1,024 reasoning tokens as the mean of the rows the crossing block committed (stuck, decided), and at a finish candidate (done; the last one of a turn that ended in calls is dropped). No extra forward: the rows are the ones the drafters already read, copied asynchronously behind the forward into pinned slots allocated once. A ring on the NVMe (~/.qwen38-spark-engine/probe/snapshots/<day>/, records and an index line each with the journal row id, the session, the class and effort, the point and the outcome keys; no text), written by its own thread after the request ended, journaled (kind = probe), at most 100 MB a day, 30 days, pruned at start and at 04:00, refused inside a repository. Night 2's traffic writes 81.1 MB (1,023 TandemLLM requests with tools, 271 think points, 17 done claims, 61,860 bytes a point with its index line). X-Tandem-Probe: off takes a request out (the live A/B of tools/probe_measure.py).
• tools/probe_train.py: fit joins the snapshots to the journal's later rows (the class Qwen decided in its no-think or in-context answer and the 216-prompt fixture's labels; the D1 guard's acts for stuck; the next checker RESULT for done), fits a logistic probe per question held out by session, and reports, beside the class accuracy, the source-confound numbers (arXiv 2606.02907): the accuracy of a probe predicting the source from the same rows, the class from the source alone, the deconfounded accuracy (each source's mean row removed) and leave-one-source-out. Gates: class 90 % on the deconfounded split, stuck precision 0.9 at the guard's act rate over 30 positives. fixture sends the 216 prompts through a server with the snapshots on (one output token each, GPU), growth reads a night's bytes from the ledger.
• The probe stage (classifier.probe_stage in the router document: off, shadow, on; a probe file and accept 0.9). Shadow: the request's own prompt snapshot is answered after the request into extra.probe. On: where the plan decides under the engine lock (the in-context stage's follow-ups and long prompts), the probe answers first on a prefill of the turn's prompt up to its insertion point, and the question is asked only below accept (stage and source probe). One answer is 17 us on the box CPU (6 x 5,120 rows, 9 classes). A probe of other weights, width or rows is refused. Contract: the router schema and the decisions schema (stage probe, stage_ms) are additive.

What changes on deploy: nothing in what TandemLLM answers. serve.env gains QWEN38_PROBE_SNAPSHOTS (off unless the deploy runs with SNAP=on) and QWEN38_PROBE_DIR; the stored router has no classifier.probe_stage, so the stage is off. Turning the snapshots on is the serve.env key and a restart; the stage is a router version. A rollback to 0.3.11-test leaves the ring on the disk and refuses a router version saved with probe_stage (it serves the version before).

0.3.11-test, 2026-10-02

0.3.10-test 468d669 plus P1 of "One engine for all models" (branch tandem-n1): MiniCPM5-2B inside the engine as the narrator that writes the state lines, off by default; and a per-turn-kind effort for tool-result turns (branch tandem-f2), inherit by default. The engine's decode path and ops/ are 0.3.10-test's; serve.env gets two keys (QWEN38_NARRATOR_MODEL, set to minicpm5-2b by the deploy, and QWEN38_NARRATOR_DIR), and nothing loads at the start. With ui.narrator off, which is what every stored profile gets, TandemLLM's output is 0.3.10's and raw is byte-identical.

• A small-model slot in the engine (engine/generative/): a generative model with its own tokenizer, weights and KV cache, loaded through the residency planner like the decision models (its own memory pool and stream, the planner's check for its role, never during a hold or beside another load, its measured MemAvailable drop journaled as loaded, three failures in a minute unload it), and run inside a time budget: the deadline is checked before every token. The first model is MiniCPM5-2B (openbmb/MiniCPM5-2B, a LlamaForCausalLM, BF16): the forward is our own plain PyTorch Llama (engine/generative/llama.py, eager, greedy, 1,024 cache rows, the longest common prefix of the last prompt reused), no transformers model, no SGLang or vLLM. Its greedy ids equal transformers' on the real checkpoint for 12 of 12 narrator prompts (box CPU, BF16, tools/narrator_reference.py), and its logits equal transformers' on a random tiny checkpoint in fp32 (tests).
• The narrator (server/conductor/narrator.py, profile ui.narrator, off by default). With the state lines on and the narrator on, each line is the narrator's: the step's kind, its meaning and its facts in words go to the model, which answers with one line of at most 14 words and 24 tokens inside the narrator role's latency budget (150 ms); a request spends at most four budgets. Anything else -- not loaded yet (the first line starts the load), refused, out of time, at the token cap, too many words, markup, an error -- is the template, journaled with the reason (extra.states[].summary and .narrator). It runs on the request's thread when the line goes out (before the prefill, between two of Qwen's rounds, after the last token), Qwen waiting, never beside Qwen. A narrated line ends in a WORD JOINER (U+2060) before the frame's blank line, and the strip takes such a line of any text, so Qwen never sees one either. The roles document's narrator.mode (on, shadow: journal only, off) and budget apply without a restart. Routes: GET /v1/conductor/narrator/status, POST .../load|unload, POST .../try. The Profiles screen has the switch under "What the user sees".
• The role contract knows a generative kind: window between_rounds (a generative role only, at most 1,000 ms a call), writes state_lines (a generative role only; the honesty rule stands: only the reasoner writes client-visible tokens), contract generate. A roles document stored before the narrator gets the built-in narrator role (journaled as a default). The residency document claims 6 GiB for it (declared, in the order after the decider); the engine's ceiling leaves its bytes out as it leaves the decision models'. The session monitor's tool-check event carries the calls' tool names and file paths to the state lines (not to the journal). Contract: additive (roles, profiles, seeds).
• The effort of a tool-result turn (F2). The router document's structural.tool_turn_effort (inherit, off, low, medium, high, xhigh; a profile's thinking.tool_turn_effort over it) gives a turn whose newest message is a tool result, SRV-161's tool media included, that level's thinking switch and cap (low: 1,024 by the hidden close); the first turn and every new user message keep the class's, the note stays the class's (the prompt is the same token for token), a client's own number still wins, a checker FAIL raises the level as before. Journaled as extra.effort_source: tool_turn with extra.class_effort. inherit, the default and what every stored document gets, plans as 0.3.10 (the test session's 18 turns plan and render identically on 62d9f5a and the branch). Routing › Stickiness and structure and Profiles › Thinking edit it. Built to test why spiral reached PASS fastest when its tool turns ran at chat/low by the SRV-161 bug (eval arm f: profile overnight-f, low). Contract: additive (router, profiles, meta).
• tools/narrator_measure.py: the n = 1 live check (night 2's recorded events through the narrator at its budget and with the budget lifted, latency p50/p90, and Qwen's decode rate with and without the narrator writing lines, gate 3 %). tools/narrator_reference.py: the forward against transformers.

What changes on deploy: nothing in what TandemLLM answers (no stored router or profile has tool_turn_effort, so every tool turn keeps the class's effort). serve.env gains QWEN38_NARRATOR_MODEL=minicpm5-2b and QWEN38_NARRATOR_DIR; the server then serves the narrator role but loads nothing until a profile with ui.narrator: on asks for a line (MiniCPM5-2B is on the box under ~/.qwen38-spark-engine/models/minicpm5-2b, 5.03 GB). Turning it on is a profile version, no restart. A rollback to 0.3.10-test refuses a roles version saved with the generative kind or a router or profile version saved with ui.narrator or tool_turn_effort (0.3.10's schema does not know them): it serves the newest older version and shows the error in GET /v1/conductor/config.

0.3.10-test, 2026-10-02

0.3.9-test 94f129b plus two features of branch tandem-g1, both off by default: Qwen decides inside the conversation it already holds (the in-context stage), and TandemLLM's own steps shown to the user as lines in the reasoning (ui.states). The engine's decode path and ops/ are 0.3.9-test's (engine/cache.py gains checkpoint_upto, which only the stage passes), serve.env gets no new key. With both off, which is what every stored router and profile on the box gets, TandemLLM's output is 0.3.9's; raw (X-Tandem: raw) is byte-identical. One thing runs on every TandemLLM request from the deploy: the handler takes framed state lines (\n› <template text>\n\n) out of the assistant messages a client sends back, which changes nothing for a request that has none.

• Qwen decides with the conversation it already holds (the in-context stage). A follow-up (a user message after an earlier answer) or a prompt of at least min_tokens (8,192) is classified by the engine itself: under the engine lock, right before the turn's own prefill, it prefills the turn's prompt up to its last message (the session's notes replayed) plus a hidden question with one letter per class, thinking off, and takes the class from the last row's logits over the letters (one token each, nothing decoded). Laya keeps short first messages. The turn then resumes at the grid line below its insertion point, so it re-reads at most a grid chunk plus its note and tail; its logits and state equal the turn without the stage bit for bit (tests on the random model, including the drafter's rows and the anchors). The stage takes state-store snapshots only at lines up to where the turn parts from it and no session snapshot; nothing of the question streams, counts in usage or reaches the turn. Journaled as stage qwen_context (label, confidence, probabilities, ms, its prompt tokens, prefilled and reused) and source qwen_context. Below its threshold, failing or not run, the session's class. Switched by the router document's classifier.context_stage (mode off by default, min_tokens, followups, accept, question); off, nothing changes and the prompt is never counted. Images in the conversation go through the stage expanded as the turn's prompt expands them (the same rows, rotary coordinates and cache keys), so opencode's sessions with screenshots are classified in context too, and the turn after the stage is bit-identical (test on the random vision model). cache.prefill gains checkpoint_upto. Contract: the decisions schema allows the stage and the source (additive).
• TandemLLM's steps shown to the user (ui.states in a profile, off by default). Each step the conductor takes is one short line in the reasoning the client already renders, at the moment it happens: deciding: code_edit, high effort, retrieving: skill photo-to-3d, compaction seen, steering: gate progress, thinking closed at the 16,384 cap, loop detected, steering, empty answer, continuing, checking the tool call, repairing the tool call, asking for the tool call again, giving up (reading memory: N notes is defined for later). Templates, no model, no latency; a summary field per line for a clerk later. The marker is › and a space, framed as "\n› text\n\n"; in reasoning_content a delta, in tags inside <think> (a step after the block closed is left out of tags); a non-streamed answer carries them in its reasoning. Whenever the conductor is configured, the server strips every framed template line from the reasoning and content a client sends back (opencode returns its reasoning as reasoning_content), so the prompt and the cached prefix are what they would be without the feature (tests: the same prompt ids on the checkpoint's template, the same engine prompt and usage through the handler). Not in usage, the suffix store or the response cache's key; the journal has extra.states and extra.states_stripped, the transcript a ### states block. ui.kinds turns a kind off. The Profiles screen has the switch. Only for a profile in mode on and n = 1; off, raw and shadow stream every byte as before.

What changes on deploy: nothing in what TandemLLM answers. The router document on the box has no classifier.context_stage and loads with the stage off; no stored profile has ui, so every profile's state lines are off. Turning either on is a config version (Routing › Qwen in the conversation, Profiles › What the user sees), no restart; ~/c310-live.py turns them on for its checks and restores the versions it found. A rollback to 0.3.9-test refuses a router or profile version saved with the new keys (0.3.9's schema does not know them): it serves the newest older version and shows the error in GET /v1/conductor/config, as the 0.3.9 rollback note says for roles.

0.3.9-test, 2026-10-02

0.3.8-test plus stage R1 of the roles and harmony design (VIS-E7, branch tandem-r1, built on the night of 2026-10-01 from a17e99f) and the client thinking budgets (branch tandem-budget, 2026-10-02), both merged onto 0.3.8-test 54f5c98. The engine's decode path and ops/ are 0.3.8-test's, serve.env gets no new key. R1 adds no step to the path that produces TandemLLM's answer (the transcript writer reads the request after it ended, on its own thread); raw (X-Tandem: raw) is byte-identical and is never seen by the writer.

• The role contract (SRV-133). A role in the roles document declares kind, memory_claim_gib (renamed from memory_gb), window, preemptible, latency_budget_ms, fallback, tokenizer_sha256, writes and sharing; a role without a claim or a window, a client-visible writer that is not the reasoner, in_decode for a kind other than a decider or a probe, and token sharing without the engine's own tokenizer hash are refused. The live v2 roles document loads with defaults, journaled once per start; a restore of an older version writes it whole. The registry checks each decision model against the same contract (Laya claims 4.4 GiB, the first checkpoint's measured MemAvailable). The Profiles screen edits every field with the reason an option is off; the Deciders screen shows each model's contract. Contract: memory_gb is now memory_claim_gib (old versions load), the rest additive.
• Session transcripts on the NVMe (SRV-137): one markdown block per TandemLLM request with tools under ~/.qwen38-spark-engine/memory/sessions/<yyyy-mm>/<session>/ (transcript.md and meta.json), compaction blocks, counters only for chats without tools, the secret-pattern refusal on every block (counts per kind in the journal, never the value), 0600, 30 days with keep, settings.memory.transcripts versioned like the rest. Written by its own thread after the request ended. Only the :8000 service's conductor directory (~/.qwen38-spark-engine/conductor) writes there; any other conductor directory keeps its memory inside itself, a memory directory inside a git work tree is refused, and a server without the conductor (every benchmark row) writes none.
• The Conductor view's Memory screen (VIS-48, VIS-49): the sessions list with a keep switch, a transcript viewer (200 blocks a page, hidden notes and acts highlighted with a jump to their journal row, compactions folded), Markdown and JSON export, the transcript settings; /v1/conductor/memory/* needs the admin token for reads too. Phone layout: cards and a full-width viewer.
• Client thinking budgets in every request shape, and an answer reserve (server/thinking.py). The server read max_reasoning_tokens and thinking_budget and nothing else, and took reasoning_effort as a template word: a client that sent its budget another way got no cap, and a turn whose reasoning ran into max_tokens ended inside the block with no answer (the Hermes agent sends reasoning_effort: medium; twice it reported "Thinking Budget Exhausted"). Now OpenRouter's reasoning object, Anthropic's thinking object, the budget numbers in chat_template_kwargs and under other servers' names, the effort words as their levels' caps (minimal and low 1,024, medium 8,192, high 16,384, xhigh and max none) and every off form are read into one budget, with the precedence in docs/server.md; a client that asked for one gets its reasoning closed early enough to leave --answer-reserve tokens of the output limit for the answer after the closing phrase (auto: min(2,048, a quarter of the limit)). The close is the monitor's hidden one where it acts (journal task client or reserve), ThinkBudget's streamed phrase otherwise. Under TandemLLM's router a client's number wins and its effort word yields to the turn's level. This changes the output only for a request that sends one of these fields: a client's own max_reasoning_tokens under an acting monitor now closes hidden instead of streaming the phrase, and every client that sends an effort word (Hermes sends medium by default, an opencode model with reasoning_effort in its options sends it on every turn) gets the reserve and, outside the router, the word's cap. A request with none of them, and raw, are byte-identical.
• The chat template's 500 on effort words (the budgets branch). The template renders only low, medium and xhigh and raised on any other word, so reasoning_effort high, none, minimal or max (what Hermes sends for its /reasoning levels and for thinking off) was a 500. Now minimal renders as low, max as xhigh, high leaves the template at the server default and is the cap alone, and none renders the template's own thinking off (byte-identical to enable_thinking: false); an unknown word is a 400 naming the field.

What changes on deploy: the stored roles version (v2) has no contract fields and loads with the defaults (window idle, preemptible, fallback PASS, writes nothing, tokenizer none, sharing text, the budget its timeout, the kind from its name), journaled once per start; nothing is written to the roles document until it is saved. The settings document has no stored version on the box, so the seed applies (transcripts scope tools, 30 days, clip 4,000 characters): from the first TandemLLM request with tools after the start, transcripts accumulate under ~/.qwen38-spark-engine/memory/. R1's gate is one night of transcripts with 0 bytes written outside ~/.qwen38-spark-engine/. A rollback leaves the memory directory in place (0.3.8-test does not read it). The budgets add the --answer-reserve flag (auto by default; QSE_ANSWER_RESERVE), not a serve.env key: from the deploy, a request that sends a budget field (a number, an effort word, an on or off form) gets the reserve, and on the raw name its effort word's cap.

0.3.8-test, 2026-10-02

0.3.7-test plus the tickets of VIS-E7 built on the night of 2026-10-01 (branch tandem-night2: tandem-night-tickets with SRV-157, SRV-153 and OPS-49, then SRV-159 and the eval tooling). The engine (engine/) and ops/ are 0.3.7-test's, serve.env gets no new key. What TandemLLM serves changes in one place on purpose, the effort levels' caps (SRV-159); effort by task size stays in shadow (no stored profile has thinking.by_size), the classifier's context line stays off, xhigh stays uncapped, so 0.3.8 measures like 0.3.7 apart from the caps. Raw (X-Tandem: raw) is byte-identical.

• Real thinking budgets per effort level (SRV-159). Before, medium, high and xhigh all had budget 0 and differed by their sentence alone. Now low caps reasoning at 1,024 tokens (as before), medium at 8,192, high at 16,384, xhigh has no cap below the D2 backstop (as before), off stays thinking off. This changes TandemLLM's output on turns at medium and high: today that is research (medium) and plan, design_ui, drawing and three_d (high); code_edit and debug stay xhigh, uncapped (with effort by task size on, a small code_edit task thinks at high and gets the 16k cap). The cap closes with the session monitor's hidden close (SP1, as the D2 backstop does): the vendor's phrase goes into the model's context and the client receives </think> and the answer; low's close, which streamed the phrase before, is hidden the same way. Without a monitor that acts, or for n > 1, ThinkBudget's streamed phrase closes it as before. A profile's thinking.budgets overrides a level (Profiles › Thinking); the Routing screen edits the levels; both are versioned. A client's own budget still wins and raw is untouched. The router document on the box has no stored version, so the new caps apply on deploy; restoring them is a router version with the old values.
• A tool's media is a tool-result turn (SRV-161). opencode sends an image a tool returned (a read of a PNG, a browser screenshot) as a user message right after the tool results, captioned Attached media from tool result:. The router took it for a new user message: Laya was unsure, Qwen said chat at 0.96, and the session switched to chat at low effort (1,024-token cap) for every tool turn after it. Read from the ledger: in every photo3d TandemLLM session of eval night 2 and round 1, 3 to 5 turns ran as three_d and about 90 % as chat/low (14 to 21 turns a session at the 1,024 cap); spiral the same after its first code_edit turns. Now such a message (after tool results, media parts, no text but the caption) keeps the session's class and asks no model, journaled kept: tool_media; a person's message with an image is classified as before. This changes TandemLLM's output on the image suites: their turns think at their class's effort (three_d and drawing high, 16,384 cap) instead of chat/low. Whether that changes their pass rate is untested (round 1's photo3d passed with the old routing).
• Effort by task size (SRV-157). The class sets the ceiling and the session's size picks within it: code_edit thinks at high while its task is small and at xhigh once it is medium or large. The size comes from four structural signals (the prompt tokens the session added after its first request, the files its tool calls named, the most gates a checker's RESULT line named, the turns), each with two thresholds in the router document's effort.size (first guesses from eval round 1: 20k/60k tokens, 3/8 files, 4/8 gates, 30/100 turns); it never falls within a task, and a compaction starts it over. The profile's thinking.by_size (off, shadow, on) says whether it acts; shadow is the seed and what every profile stored before it gets, so a deploy changes no effort until a profile sets it on. The journal keeps the size, its signals, the ceiling and the effort per turn (extra.size, shadow.size). The Routing screen edits the levels per class and the thresholds, the Profiles screen the mode, the test box shows the size. tools/tandem_eval.py reports turns per effort level per run and per suite and arm (the efforts and size_mode columns; the Compare screen shows them).
• The classifier's digest is sized per stage (SRV-153): the decider's to its own limits, head and tail. Before, the decider cut the 2,000-character digest from the front and never saw a long message's tail; on the new long set Laya's best guess went from 29 to 33 of 60 right (requests at the tail 7 to 14 of 20) and its confident wrong answers from 3 to 0, the short set unchanged. The context line ([email protected] tools=read,edit,bash files=.py,.md checker=calccheck:FAIL compaction=no, at most 120 characters out of the tail's share, never a path or a file's text) is built and switched by the router document's classifier.context_line, off: with Laya alone it failed the ship rule (short set 50.0 % to 32.4 % right at 0.80; follow-ups that change the task 10 to 1 of 12, pulled to the previous class). The Qwen stage with the line is not measured yet. tools/conductor_eval.py compares digest variants (--digest today,sized,line) on several sets (--set repeated), reports the router's stickiness as its own stage, and can load the decision models in process on the CPU (--local-deciders laya-english --device cpu) without touching a server. New labelled sets: 60 long prompts of 1,600 to 7,200 tokens (five classes, the request at the head, the tail or the middle) and 66 short follow-ups whose class the session decides (tests/fixtures/conductor/make_sets.py writes both).
• tools/idle_gaps.py (OPS-49, Harmony R0): the idle gaps between a session's requests from a copy of the usage ledger and opencode's steps (names and times only), per tool kind, as a histogram with percentiles, gaps over 5 and 20 s per hour, the idle time per session hour and what fits in the gaps. Read-only; the live ledger is refused.
• tools/conductor_eval.py counts one way (night plan A2): per model stage all right (the best guess, threshold ignored), confident right (right and at or above the threshold, over all prompts), coverage and confident precision, with the threshold and the errored answers (NOT A COUNT when any); --rescore REPORT --at 0.6,0.8 re-counts a kept report. The morning's 63.9 % and the night's 50.0 % for Laya on the 216 prompts were the same answers counted at 0.60 and at 0.80.
• tools/tandem_eval.py --resume (OPS-63) continues an interrupted night: only what has no record runs, an errored record is kept aside and run again under a new tag, the store is cleared first, night.json keeps the night's build and each resume, and the report says so (resumed).
• The box keeps a suite report whose night carries a suffix (2026-10-01-r2; 0.3.7-test refused it with a 422), and GET /v1/conductor/evals marks a report a later one supersedes (superseded_by). The Compare screen shows turns and thinking tokens per suite and arm, labels a superseded report and shows the newest one nothing replaced first.

0.3.7-test, 2026-10-01

0.3.6-test plus the engine's memory limits (OPS-47), the narrow drafter arm as a switch (SPD-75), branch tandem-cache-limits, and the eval reader fix with a fresh-text live_check (OPS-63), branch tandem-evalfix. The design decisions of 2026-10-01: a 70 GiB ceiling, a 77 GiB declared peak, capped caches, a parked session; the context window stays 262,144.

• One memory policy for the serving caches (engine/cache.py Budget). Between requests the engine holds at most MEM_CEILING_GIB (70), allocated plus the allocator's reserve; during a prefill at most MEM_PEAK_GIB (77). Before a prefill and before the response starts, the free blocks go back to the driver and then the caches, cheapest first (state store, body anchors, guest stash, parked session, pinned head), until the prefill's workspace fits; if it still does not, the request gets a 503 with Retry-After instead of running the box into its out-of-memory wedge. Every cache allocation asks first and may take the room of the cheaper ones. After a request the reserve over RESERVE_SLACK_GIB (2) goes back to the driver. Measured in a hold: steady state 67.25 GiB with every cache filled, a 261,611-token prompt peaking at 68.35 GiB allocated, nothing shed.
• The caches' caps: resident anchors 2.5 GiB (17), guest stash 1.0 GiB for guests up to 8,192 tokens, pinned head 4.0 GiB (no longer out of CACHE_GB), state store 2 GiB with a 1 GiB entry (room is made before the clone, not after), and the parked session, 4 GiB: when a request of another session (x-session-id) replaces the resident prompt, its rows past the shared head are kept, and that session's next turn swaps them back instead of re-reading them (measured: 41,984 of 53,209 tokens reused, 11,225 re-read in 16.4 s). A compaction drops its session's store entries and parked rows. Every limit decides only where a prefill starts; resumes from the park are bit-identical to a cold prefill (tests).
• The residency planner subtracts the engine's growth to its declared peak (77 less what it holds now) before a role's claim, not just today's MemAvailable: at 37.9 GiB free with the engine at 70, a 12 GiB reader is refused (18.9 left) where it was admitted.
• Every limit is a key in ops/serve.env (MEM_CEILING_GIB, MEM_PEAK_GIB, RESERVE_SLACK_GIB, RESIDENT_GB, RESIDENT_STASH_GB, RESIDENT_GUEST_MAX, RESIDENT_PIN_GB, RESIDENT_PARK_GB, CACHE_GB 8 to 2, STORE_ENTRY_GB, NARROW_ARM), passed by ops/start.sh. The System view's Memory plan panel shows each with what it holds now, draws the ceiling and the peak over the split bar, keeps the shed log and the last session switch's re-read cost, and edits them as the residency document's engine section: live keys apply at the next request, PREFIX_CHUNK and NARROW_ARM say they need a restart, MAX_LEN is locked. Metrics qse_memory_bytes{part}, qse_memory_shed_total{level}, qse_requests_refused_total{reason="memory"}. Contracts: additive.
• --narrow-arm off (NARROW_ARM=0, SPD-75): the block-16 drafter alone, valid only with QWEN38_LEN_MODE=wide, about 8 GiB less. Off by default. A design decision (2026-10-01 20:25): keep the narrow arm, NARROW_ARM=1 is served; the switch stays an option. The same-hour comparison is in SPD-75.
• tools/mem_gate.py (the 262k memory gate for a hold), tools/verify_spec.py --narrow-arm, tools/forced_bench.py _nn configs.

From tandem-evalfix (tools only, nothing the engine serves changes):

• tools/tandem_eval.py reads a run right (OPS-63). Round 1 (2026-10-01) published spiral and photo3d as failed on both arms, with 0 tokens, 0 turns and one needed continuation on every run; a separate checker's runs on the final shots passed all four. The causes: opencode export was read through a pipe, and opencode 1.18.33 exits before a pipe has taken more than 64 KiB, so every export ended at 65,536 bytes, did not parse and read as nothing (it now goes to a file; an export that cannot be read gives null counts and says why, never 0); the image suites had no check of their own, so with nothing read no gate passed (spiral and photo3d now check the last stage's shot, final_shot in the suites file: s4-animated.png, and final.png or the highest of S7/S6/S5 in any case, never a heatmap, an hourly shot or s4-mid.png; the check's metric lines are kept with its verdict); and the continuation count was taken before the final check, so a passed run counted one (now: one each time opencode ended in time with the suite not done).
• --report DIR --reread rebuilds a night from its run folders, the sessions and the engine ledger, keeps the night's own build (--build, else the night's report.json) instead of the build the server serves by then, names the published report it replaces (--supersedes), and keeps the replaced report and run records beside the new ones. The report's markdown gets a per-run table. Schema: optional reader, final_check.shot and metrics, supersedes, superseded_by.
• tools/live_check.py sends new text on every run: four seeded stories with the run's nonce in the prompt and in the answer (--seed repeats a run). Its fixed prompts had been learned by the persistent suffix store, so on 0.3.6-test no request decoded long enough for the live decode-rate check.

0.3.6-test, 2026-10-01

Stage A of the decision models inside the engine (ENG-179, SRV-131, VIS-46). Off by default (QWEN38_DECIDER_MODELS empty): nothing of it is imported and every response is the release's.

• The engine serves decision models itself (engine/decision/): Laya's three checkpoints through transformers' ModernBERT with the Laya SDK's own arithmetic (fp32 parameters, autocast bf16 on CUDA), equal to the SDK at 0.00 on 22 questions (tools/decider_equivalence.py, box CPU and GPU). Each model gets its own CUDA memory pool and stream; a forward runs only while no request holds the engine; each caller gets a forward of its own (QWEN38_DECIDER_COLLATE=rows collates callers, which moved a probability by up to 3.4e-3 on the GPU); a model whose forward fails three times in a minute is unloaded. Loads go through the residency planner (claim 8.2 GiB, floor 15 + margin 6 GiB, never during a hold or beside another load); ops/start.sh asks for the on_start ones after the roles.
• POST /v1/systemone and /v1/predict on the engine port (arbiter's Jev contract, the admin token or the box itself), decision_models beside data in /v1/models (TandemLLM stays the one chat model), GET /readyz.
• The conductor's decider role asks in process (endpoint: self, the new seed, or engine:laya); the deciders document says each model's load policy and which model answers classify and packages; a change of it re-asks a cached classification. /v1/conductor/deciders/* and the Conductor view's Deciders screen (models, Load and Unload with the planner's reason, the model per question, the memory row, a test box). The System view's memory plan shows the decision models as their own part.
• NOTICE: the Laya SDK (Apache-2.0, ported files say what changed), ModernBERT, mmBERT, the Jev contract, arbiter.

0.3.5-test, 2026-10-01

A test build for the box, not tagged: 0.3.4-test plus the supervisor in the Live view and the residency planner (branch tandem-p2-views).

• The supervisor where the dashboard is watched (VIS-44): the Live view's Now line keeps a line for the running request's last supervisor action ("Supervisor: closed the reasoning at 2,880 tokens (D1, score 0.93)", or what a shadow decision would have done); each request row, live and in the Dev tab, has a badge with its actions (filled) or shadow decisions (outlined) that opens a drawer with every decision from the conductor's journal; the header shows the session monitor's mode; the Conductor view's session gets one wall-clock axis with every turn, the monitor's actions, the router's notes, the compactions and the checker results, and Tier 1's shadow agreement once there is any. Live 1.2 adds shadow and the decisions' score, tier and at_tokens; the live engine block and the summary add the monitor's mode; a requests row adds conductor; GET /v1/conductor/events?request=. All additive; a server without the monitor shows none of it.
• Residency planning (OPS-41): the engine reports where its memory goes (memory.split in /v1/dashboard/system: weights, KV pool, GDN state, the drafters' context caches, vision, the state store, the resident prefix, the rest and the allocator's reserve, and each load stage's share, also printed as a [memory] line at start). The residency planner decides what may load beside the engine (floor 15 GiB plus a 6 GiB margin, claims at their peak, one load at a time, never during a hold), journals every decision, and is edited in the dashboard as the conductor's residency document; GET /v1/conductor/residency/plan and the System view's memory plan show it. ops/roles.sh loads roles in that order (none configured today), with vLLM roles held to BF16 and the planner's memory fraction; hold.sh stops the roles first and waits a settle time after a stop.
• C2's allowlist (SRV-113): the project-rules check takes allow_paths, path patterns no C2 rule applies to (a path covers everything under it; *, ? and [ make a glob; ~ is home; a relative entry is under the project root). The default is opencode's own scratch, /tmp/opencode, which opencode's tool texts send the model to and which the /tmp/ rule flagged; a monitor document saved before the key existed gets the default. It is a field of C2 on the Conductor view's Monitor screen, which shows the default for a document that leaves the key out. The policy file is version 4.
• Changed behaviour: a streamed request with stop strings and tools no longer streams a call's arguments while the model writes them; each call goes out whole when its block closes. A stop string can end the generation inside a <tool_call> block, and the arguments already sent could not be taken back: the client was left with a call whose JSON never closed ({"filePath": "src/a.js), beside the block's text. The release 658b5fb did the same; the harness did not add it. Now such a block is the text it was, as the non-streamed answer already had it, and no call goes out. Requests without stop strings stream as before. A call the token limit cuts still streams half its arguments with the monitor raw (SRV-47); with a tool-call check able to act it gives up as before.
• The speed guard's result for a build (tools/speed_guard.py): two row3 reports of the same hour become a speed_guard report in the conductor's evals/ directory, with the build the server serves, the medians, the deltas and the verdict by row3's rule. One run a side gives no_noise_estimate: row3's rule reads every one-run difference as resolved, and the report says instead that the noise is unknown and the row decides nothing. The Conductor view's guard line and the Compare screen show the result for the serving build (a note for a one-run short gate, with mean, TTFT and ms a block; a warning for none, a resolved-worse one, or one recorded for other code) instead of "speed guard not run" for every build.

0.3.4-test, 2026-10-01

A test build for the box, not tagged: 0.3.3-test plus the session tasks and the eval harness (branch tandem-p2-session).

• Session-level tasks in the session monitor (SRV-121, docs/server.md): after opencode's compaction, if the first two tool-call steps read none of PROGRESS.yaml, AGENTS.md or a skill, the next request gets a hidden re-read note (S2, once per compaction); a checker gate that fails three times in a row without getting 10 % closer to its limit gets a note naming the readings and the stage rule (S1, once per gate and stage); no file changed, no commit and no better checker number for 20 minutes gets a no-progress note (S4). Each is one hidden note at the end of the prompt, at most one a request, silent without tools or without a state file in the system prompt. With --conductor on and the shipped policy all three act; under the conductor's profiles only overnight runs them.
• A client that has gone: no request for 20 minutes after a turn ended is journaled as client_gone by the monitor's watch (--conductor-watch-s, 30 s), and pushed to the phone through ntfy only where notify.enabled is on: off in the policy file, on in the overnight profile. The token is a file on the box (~/.qwen38-spark-engine/conductor/ntfy.token).
• GET /v1/conductor/watch (what the session tasks hold per session) and POST /v1/conductor/evals (a suite report kept for the Compare screen). The Conductor view's Monitor screen gets a session tasks section and the watch; the Profiles screen's Context section sets S1's runs, S4's minutes and the push; the Compare screen shows the newest suite report with its verdict, n and the indicative label.
• tools/tandem_eval.py (OPS-43): TandemLLM against the raw engine (X-Tandem: raw) on the same build and night, arms alternating, a unique tag and a fresh project copy per run, the prefix store cleared between arms, checkers hashed before and after, a watch-only driver; suites in bench/tandem-eval/suites.json (two short coding tasks with checkers, the spiral portrait, photo to 3D, pagoda seasons when a clean template exists, the client smoke set). The report (docs/contract/dashboard-v1/tandem-eval.schema.json) gives pass rate, minutes to PASS, tokens, decode tok/s, empty stops, continuations, interventions and the cached share per suite and arm, with n and the verdict.
• The checker parsers know calccheck and taskcheck (the eval suites' coding tasks).

0.3.3-test, 2026-10-01

A test build for the box, not tagged: 0.3.2-test plus the session monitor's tool-call checks and one journal for the conductor (branch tandem-tools-obs), and the one-model naming on every surface.

• Every public surface names the one model: with the conductor configured, /health, its cache block, /v1/dashboard/system, the Live stream's engine block and request rows, and every completion response say TandemLLM, whatever name the client sent. /health, the cache block and the System view add base_model, the base engine's own name (--served-model); the metrics keep it as their model label. Without the conductor nothing changes. The ledger's model column stays the name the client sent.
• Tool-call checks in the session monitor (SRV-113, docs/server.md): with one able to act, an answer's tool calls are held to its end token and checked before any goes out. A block the parser cannot read is repaired when the fix is not a guess, else the model writes the calls again after a hidden reminder with the parser's reason (two times a request, then content as before); the same call a third time after two identical results is written again too ("do something different"); risk floors (rm -rf outside the project root, git push, writes outside the root, a download piped into a shell) and AGENTS.md rules (numbered copies, /tmp paths, files over 400 lines) are journaled in shadow; a call the token limit cut gives up rather than going out as half a call. Every pattern, limit and reminder is in the policy file, and each profile turns the four packs on or off. With the shipped policy and a tool-call task able to act, a call arrives whole at the end of the answer instead of streaming its values; with the monitor raw nothing changes.
• One journal for the conductor (SRV-115): the router's decisions and the monitor's in the usage ledger's file, table conductor_events with its texts in conductor_texts (ledger schema version 2, migrated in place at start; 30 days; left out of the weekly backup). The router's earlier decisions.sqlite3 is copied in once. The ledger's requests gains conductor_acts, conductor_hidden_tokens and conductor_acted, and the dashboard's speed figures leave acted requests out. /metrics gains qse_conductor_* (decisions with passes, actions, hidden tokens, gave-ups, tier 1 latency and failures, reasoning tokens cut). Contract live 1.2 adds activity.conductor (with supervising while a hidden forward runs). GET /v1/conductor/events[/{id}], and the session export lists every hidden text of router and monitor, "N interventions in M turns". The monitor's policy is the conductor's monitor document, and the dashboard's Conductor view gets a Monitor screen: the journal, and every rule rendered from the policy's schema.

0.3.2-test, 2026-10-01

A test build for the box, not tagged. It carries the conductor, the session monitor's frame and the Live view's rates.

• One model: with the conductor configured, the server serves TandemLLM alone. /v1/models lists only TandemLLM; the raw engine's name (qwen38-spark-engine) is a hidden alias that answers on /v1/models/{id} and runs every request as TandemLLM, as does any other model name. Raw is a header: X-Tandem: raw, or the locked raw / bench profile in X-Tandem-Profile, serves the release's path byte for byte. tools/client_smoke.py and tools/session_replay.py gain --x-tandem raw, and tools/row3.py --no-server refuses a TandemLLM server (the atlas runner cannot send the header). The ledger's model_served is TandemLLM for every request. Changed behaviour: a client that sends qwen38-spark-engine now gets the conductor under its profile; before this build it got the raw engine.
• TandemLLM (--conductor-config, off by default): per-profile routing of each turn by task class, decided by arbiter and then Qwen with thinking off, which sets the thinking budget, thinking off (a tail splice, the head untouched), and a hidden note (effort sentence, a skill or package, rules) at the end of the prompt, replayed on later turns so the resident prefix holds. Versioned config with hot reload, restore and validation under /v1/conductor/*; a decision journal and a session export; the dashboard's Conductor view. Every response carries the X-Tandem and X-Tandem-Profile headers. The usage ledger gains model_served, profile, profile_version and conductor_mode, added in place.
• The turn after a long unrelated request resumes from the conversation's head. opencode's compaction request (no tools, no system text) used to replace the resident prompt, and the next turn, which starts with the same tool block and system text, re-read everything. The head of the old prompt is now pinned before such a request cuts it (--resident-pin-gb, 4 GiB out of --cache-budget-gb), and anchors every 4,096 tokens up to 32,768 are kept for it. The resume is bit-identical to a cold prefill.
• The state store keeps one boundary per prefill, the longest under its per-entry cap (--store-boundaries, 0 for every boundary). A cold prefill no longer evicts everything else with snapshots of its own prefix.
• The persistent suffix store appends only what a request adds to one of the last four prompts, with 8 tokens of context, instead of the whole prompt every turn. /health reports the rebuilds' total time. A store written before the tokenizer fingerprint gets it recorded.
• Less host work before each prefill: the lookup index and the prefix hashes extend the previous prompt's instead of starting over, and a chat prompt is tokenised a message at a time with the messages earlier requests had looked up (--prompt-token-cache). The ids and the drafts are the same as before.
• A declined session snapshot says why in the log, and declined_big counts the size refusals.
• The Live view's prefill figure shows the prefill running now, not the last finished request's end-to-end rate, and "—" when nothing prefills. The live prefill and decode rates leave each request's warm-up out: prefill after its first chunk, decode from its 50th token (QSE_LIVE_* settings). The banner shows the same number as the figure, and the prefill chart gets a sample every second while a prefill runs. timings, usage, the ledger, /metrics and the finished rows keep their end-to-end numbers. Contract live 1.1 gains fields only.
• Off by default, not bit-identical to a cold prefill: --session-trim on cuts the last verified block back to the committed tokens so the session snapshot is kept.
• The conductor's session monitor, the frame of the TandemLLM second-level harness (server/conductor/monitor/, docs/server.md): --conductor raw|shadow|on and a policy file, ops/supervisor.yaml, with every task's mode, threshold and text, validated against ops/supervisor.schema.json. It acts only at four safe points, from a fixed action set, and every decision is journaled with a [conductor] log line. X-Tandem on a request lowers the mode and never raises it, and every completion response says its mode in an X-Tandem header. raw is the default and writes what the release wrote, token for token (tests/test_serve_identity.py).
• Changed with --conductor raw: no stall close. The old detector closed the reasoning block, on by default, when a 1 to 16-token pattern repeated four times or novelty collapsed; 51 of its 52 closes in the opencode database were literal runs such as 0x000000. It is now the monitor's reasoning-loop task (D1): a pattern counts only over a span of 128 tokens, and a new arm measures the share of the last 256 tokens' 16-grams already seen earlier in the block (fires at 0.7 twice in a row after 2,048 tokens: the three empty turns of 2026-09-30 at 2,112, 2,880 and 3,008 tokens, and 4 of 80 long healthy turns). A request on which the old detector fired now writes a different text; a client's own max_reasoning_tokens is unchanged. tools/stall_replay.py replays the arms.
• With --conductor on, the shipped policy: the reasoning backstop at min(32,768, max_tokens - 2,048) reasoning tokens (D2), and a request that offered tools never gets an empty stop (E1): an end token inside the reasoning becomes a hidden close, one after an empty answer a hidden user turn with a nudge, and after two tries the response is one sentence, "No tool call after two tries. Stopping this turn.", with finish stop. Hidden tokens are not in usage, not in the suffix store, and an acted response is not kept in the response cache. The [req] line of a monitored request says conductor=MODE acts=N hidden=N.

0.3.0, 2026-09-30

Tagged v0.3.0: image input. The text path is unchanged: of 13 text requests run twice, 25 of the 26 runs match v0.2.1 byte for byte, and the other is a StairCut near tie on which v0.2.1 also differs between its own two runs. The benchmark row is not worse in either view, measured in the same hour as v0.2.1.

• Image input: image_url parts (https and data: URLs) in chat requests. The checkpoint's vision tower runs as stored (BF16), its rows go into the prompt with the model's three-axis rotary, and every cache keys on the image's content. Bad images get a 400 that names the part; --vision off turns it off. An https URL is fetched only from public addresses, with redirects checked hop by hop and one deadline for the whole download (--image-https-private on lets a private network through), and --max-image-rows (65,536) caps the prompt rows of a request's images. Text requests are unchanged bit for bit.

0.2.1, 2026-09-29

Tagged v0.2.1. No decode path changed: the engine, the drafters, the router and the kernels are the ones of v0.2.0-staircut, so the generated text is the same. The paper's numbers stay those of v0.2.0-staircut (merge c111fa7).

• The dashboard login is off by default. --dashboard-login on|off, default from QSE_DASHBOARD_LOGIN, and ops/serve.env sets off. Off: /v1/dashboard/* answers without a token, even without a secrets.env, and the page opens straight into its views, with no sign-in page and no sign-out button. Anyone who can reach the port can then read the dashboard; docs/dashboard.md lists what it shows. On: the admin token or a session, as in 0.2.0, with 404 without an admin token and a 429 after five failed sign-ins a minute. To turn it on, set QSE_DASHBOARD_LOGIN=on (in ~/TandemLLM/run/local.env on an installer setup), create the token with ops/make-secrets.sh and restart. A misspelt value stops the start.
• What the login does not change: /metrics, the full /health and GET /v1/cache/stats keep their rules, and POST /v1/cache/clear still needs the admin token as a bearer header. The box's own tools still need no token.
• Dashboard contract: GET /v1/dashboard/session is at 1.1 and says login: true or false; GET /v1/dashboard/metrics is the /metrics page under the dashboard's rule, which the Performance view now reads. The contract's README lists both.
• The balanced profile (ops/serve-balanced.env) now loads the published NVFP4 set's MLP overlay and head (~/nvfp4-bf16/mlp.safetensors, ~/nvfp4-bf16/head-fp8.safetensors), not the first FP8-derived files. The FP8 and balanced profiles inherit StairCut's verify prices, measured on the full-NVFP4 profile; the router corrects their level online, but the staircase's shape for these profiles is not measured, and neither is their speed on this release.
• The 0.2.0 entry's agreement counts are corrected to the paper's classification: of 30 prompts, 17 identical up to the end of the answer, 13 that take the other token at a near tie inside the answer, 0 other differences.
• Docs: the installer's default release is v0.2.1, and the README, the installer's closing message, docs/server.md and docs/dashboard.md describe the login as it is now. The paper is published on Zenodo, and the README and CITATION.cff cite it by its DOI, 10.5281/zenodo.23045275.

0.2.0-staircut, 2026-09-29

Tagged v0.2.0-staircut: StairCut, from branch router-eng169, merged at c111fa7. The figures below are the release gate's same-hour runs. The paper's measurements of this release, on the published NVFP4 set, are in the README, and its classification of the 30-prompt agreement check is the one in docs/exactness.md.

• StairCut: QWEN38_LEN_SWITCH=1 QWEN38_LEN_MODE=wide cuts every round's tree (the wide drafter's lattice, its chain, the lookup tree, their merge) to the node count with the most expected tokens per millisecond on the measured verify staircase, priced by context class (QWEN38_STAIR_TABLES). The lookup's continuation rate on 8-token matches is a held-out prior counted online (QWEN38_STAIR_RHO), and QWEN38_STAIR_SKIP=1 skips the head draft on a copy run. All flags are off in the code; ops/serve.env turns them on. With them off, every text is byte-identical to rc10. docs/speculative-decoding.md has the rule.
• On the published NVFP4 set, the benchmark row reads 49.33 and 50.47 tok/s with the store off (50.37 and 50.60 with a clean store). Fixed block 8 reads 47.44 and 46.82 (47.71 and 46.80), and fixed block 16 reads 44.65 and 44.95 (43.80 and 44.05), all in the same hour.
• Not every text is identical. Of 30 prompts checked against plain greedy decoding, 17 are identical up to the end of the answer: 13 over all 256 tokens, and 4 more that differ only past the end of the answer. The other 13 take the other token at a near tie inside the answer, with a gap of 0.04 to 0.76 ulp, and continue from there. None differs otherwise. docs/exactness.md states the rule.
• The router learns online: draft cost per context class, calibration, copy counts. A server keeps what one request teaches for the next. On the teacher-forced bench (25 workloads), a router rebuilt for every request averages 1.4 % below one that is kept.
• New tools: tools/router_replay.py replays router policies on recorded lattices, and tools/forced_bench.py measures configurations on one reference text.

0.1.0-rc10, 2026-09-28

Not tagged yet.

• The FP8 profile (ops/serve-fp8.env): every projection is the checkpoint's own FP8, and the head is its BF16 head. It ran at 28.4 tok/s on the benchmark row. A per-shape FP8 launch table keeps every tile bit-identical from 1 to 32 rows.
• The balanced profile (ops/serve-balanced.env): NVFP4 MLPs, the checkpoint's FP8 for the recurrent and attention layers, and the FP8 head. Not measured yet.
• --fp8-head build builds the FP8 head at load, and --price-table loads the router's verify prices for a weight set.
• First seams for other models: one registry for every engine setting, the checkpoint layout in one place, one Linear interface for every weight format, BF16 checkpoints, and drafters that declare what they read from the target. A test pins the exact bytes of a tiny model's run.
• The dashboard's Performance page no longer shifts while the Live panel updates.
• The product name is now TandemLLM (working name) in the README, the dashboard and the HTTP server header. Runtime names are unchanged.
• The NVFP4 profile is unchanged: its gate against the same-hour rc9 found nothing worse, and 13 plain requests came back byte-identical.

0.1.0-rc9, 2026-09-27

Tagged. It contains rc7 and rc8.

• Live activity. /v1/dashboard/live (contract 1.1) says what each request is doing now: queued, prefilling with progress and an estimate of the time left, thinking, writing, calling a named tool, waiting for the client between two agent turns. It also says how each of the last 20 requests ended, in one sentence. The dashboard's Live panel shows all of it.
• The Playground view loads on first use, and the bundle every tab downloads went from 56 KB to 45 KB gzip.
• The stream writer now sends every tick's event. Before this fix, it skipped an event when one landed between its read and its wait.
• 44.09 tok/s (44.14 with a clean store), against 44.16 for rc8 in the same hour.

0.1.0-rc8, 2026-09-27

• The resident prefix. The KV of the last prefill stays in place, with recurrent-state anchors every 1,024 tokens, so the next turn of a growing agent conversation resumes instead of re-reading everything. Resumes are bit-identical to a cold prefill. Requests at 77,000 to 79,000 tokens answered in 2.7 to 16.7 s, where rc7 re-read everything past 13,312 tokens.
• The prefill watch. After every prefill chunk the server checks whether the client is still there, and stops the prefill when it has gone. A retry resumes from the rows already done. A streamed prefill longer than 5 s sends : prefill done/total comments.
• 44.10 tok/s, against 44.09 for rc7 in the same hour.

0.1.0-rc7, 2026-09-27

• Tool calls are read in three forms (strict XML, lenient XML, JSON) and stream as argument deltas.
• Tool-call values carry the types of the request's own schema: 150 instead of "150". On an 8-task agent run, a client that validates arguments refused 58 of 80 calls before, and 0 of 26 after.
• tool_choice works in all four modes, and required or a named function is enforced by a token mask.
• Structured outputs: response_format (json_object, json_schema) and regex or choice constraints, compiled to byte-level automata and exact under speculation.
• More of the OpenAI API: logprobs, n, min_p, logit_bias, parallel_tool_calls. Every OpenAI field is either honoured, accepted without effect, or refused by name.
• The Live panel and GET /v1/dashboard/live.
• tools/client_smoke.py: the request shapes real clients send, checked before every deploy.
• 43.89 tok/s, against 43.86 for rc6 in the same hour.

0.1.0-rc6, 2026-09-26

• tools/block_ab.py: an in-process, alternated A/B for changes that keep the output bit-identical.
• A faster 17 to 32-row tile table and a faster FP8 head GEMM, both bit-identical.
• 44.34 tok/s (45.04 with a clean store), 92 ms a round, against 43.36 for rc5 in the same hour.

0.1.0-rc5, 2026-09-26

• Sampled requests keep the draft tree (--sampled-tree det): at temperature 0.7, 28 % more tokens a round and 19 to 21 % more tok/s, exact in distribution.
• The stream decodes a window of recent tokens instead of the whole answer, 11.9 % faster over the last 1,000 tokens of an 8,000-token answer.
• The skinny kernel reads its scales in contiguous runs.
• About 43.4 to 44.0 tok/s, 92 to 93 ms a round.

0.1.0-rc4, 2026-09-25

The first release measured in two store views. Earlier rows ran with a lookup store that could hold the benchmark's own text, and this release replaces them.

• Kernels: the skinny W4A16 kernel for 1 to 32 rows, decode attention that reads the KV once, the block drafter in NVFP4, the verify and the draft from CUDA graphs, and the commit folded into the next verify.
• Trees: a 16-node tree for the narrow drafter, 24 nodes for the wide one after 32 committed tokens, verify up to 32 rows, and a deep chain for text that repeats itself. Tokens a round went from 3.3 to 4.0.
• Sampling with seeds: a seeded request writes the same text whatever the drafters did.
• Usage and timings on every response, the usage ledger, the dashboard at /dashboard/, Prometheus metrics, access tokens.
• Long context: the page cache of the weights is dropped after load, which keeps at least 50 GB free from 8k to 128k tokens of context.
• 41.58 tok/s (41.93 with a clean store), 423 ms to the first token, against 25.74 tok/s for rc3's settings on the same row.

0.1.0-rc3, 2026-09-20

Not tagged. Serving defaults for long, real work instead of a 256-token row:

• The full 262,144-token context, 32,768 default output tokens, a 3,600 s request timeout and a 240 s queue timeout.
• The state cache budget went from 24 GiB to 8 GiB, because large snapshots on top of the page cache ran the board out of memory.
• Loop guards for the thinking phase that close the reasoning block instead of cutting the stream.
• Anti-repetition penalties, off by default, with per-request values honoured.

0.1.0-rc2, 2026-09-18

• The drafter that loses the per-request width decision is released and stops syncing.
• Operations: a lock and a hold script for measurements, and a watchdog that respects a hold.

0.1.0-rc1, 2026-09-18

The first release candidate.

• The engine: the checkpoint's FP8 read in place, NVFP4 weights from a gated clip-search quantiser, the FP8 head, and fused kernels for the recurrent layers.
• Speculation: the block drafter, the lookup drafter, tree verify through the recurrent layers, and a block width decided once per request by the length router.
• The server: an OpenAI-compatible API with streaming and reasoning formats, a bounded queue, request timeouts, health, draining, a watchdog and a service unit.
• Serving caches: session and prefix state, a persistent suffix store, and an opt-in response cache.