All releases

TandemLLM 0.14.2 2026-10-23 documentation

TandemLLM 0.14.2 fixes what an outside test of Ornith found, and the bugs we met on the way. With thinking off, reasoning that a model writes into its answer anyway now goes to reasoning_content. Ornith answers up to its card's 131,072 tokens when a request sets no limit, and at 131,072 tokens and above it decodes plain instead of drafting slower than plain. The Setup Guide no longer offers Ornith parallel chats its engine cannot run, and it sets the thinking effort for every model. cache_prompt: false reads the whole prompt and leaves the caches as they were. The lookup corpus is built for each model's tokenizer after its download. tandem-dash serve starts without the engine installed. No fingerprint moves: every model writes the same tokens as in 0.14.1 and keeps its saved conversations. Every change is in CHANGELOG.md, and the PDF below is the manual.

Answers

• With thinking off, the text before the first </think> that stands alone on its line, outside a code block, goes to reasoning_content, and usage counts it as reasoning (#93). This applies to every model, and only when no <think> of the model's own came before the tag. A block the model opens at its start is read as reasoning, and an empty pair is dropped. Streamed, the text before the tag has gone out already, so only the tag line is dropped. A tag inside a code block or in a line of text stays, and so does every tag after the first. The report's 4 long contest answers wrote everything into content on 0.14.1. Replayed on Ornith 4.0, two now carry their working-out in reasoning_content (1,536 and 68,172 characters) and only the answer in content, the empty pair is gone, and the block opened at the start reads as reasoning. Flint's answers to 10 greedy asks with thinking off are byte for byte those of 0.14.1. Ornith gets no thinking-off line like Flint's: one cleaned 3 of the 4, but it cut the median ordinary answer from 506 to 362 tokens greedy.
• A family whose template reads one system prompt first gets its thinking-off line at the end of that prompt. A developer message there made Ornith's template answer 500 (#93).
• A request with no max_tokens on Ornith's profiles can write up to 131,072 tokens, the limit its card evaluates with (#96). Every other profile keeps 32,768. The longest answer in the report, at effort xhigh, was 39,046 tokens. /v1/models shows the value as tandem.max_output, and a request's own max_tokens still wins, clamped to the window. A box the Setup Guide set up for Ornith before 0.14.2 keeps the 32,768 in run/setup.env until the field is changed. docs/setup.md says how ("The answer limit").
• When several requests arrive together as a fresh server's first traffic, none answers 500 with "no tool-call format 'xml'" any more (SPD-185). The tool-call formats were marked found before their modules had registered, and with parallel slots three of four requests in such a burst failed. The formats are now found once, under a lock.

Ornith

• Ornith's price file has the measured cost of the MTP head's backlog, the prompt rows the head reads before a turn's first draft (#110). Before, a fresh server's first long turn always drafted and paid it: 87,717 rows in 4.6 s at 112k tokens, 109,443 rows in 10.2 s at 220k. Greedy, one run each: at 112,068 tokens the turn now takes the plain step at 25.11 tok/s, where drafting ran at 18.04 against 25.16 plain. At 220,038 tokens it runs at 18.85, where drafting ran at 10.95 against 18.89. At 3,525 tokens it still drafts, 40.9 against 37.4 plain.
• The Setup Guide stops Ornith at one parallel chat and says why under the stepper (#104). Its engine builds no side slots yet, so the server ran one slot while /health showed the count asked for. Each model.yaml names its engine family, and the family's ServeSpec.side_slots says whether it builds side slots. With --parallel N on such an engine, /health's parallel names the one slot and why. The other families keep the engine's 16.

Caches and drafting

• cache_prompt: false on a request is honoured as a cold read (#98). It reads the whole prompt (timings.cache_n 0), uses and writes no state-store entry, disk-tier anchor or end anchor, and the response cache neither answers it nor keeps its answer. The conversation in place is set aside as for another client's and is where it was for the next request. true, a missing field and any other value keep the caches on. In the report, 32 of 37 requests that sent false had reused cached tokens. docs/server.md has a cache_prompt section.
• The lookup corpus is built for the served model's tokenizer after its download (#97). One store a tokenizer sits under CORPUS/tokenizers/<16 hex of the tokenizer's sha256>/. ops/use-profile.sh and the Setup Guide's install build it once the weights are complete (about 31 million tokens, from one English and one German Wikipedia shard and local Python sources). install.sh --setup downloads the two shards (1.2 GB) and sets CORPUS, and ops/start.sh passes the engine the store of the served model's tokenizer. A failed build logs a line and the switch goes on, and QSE_CORPUS_BUILD=off skips it. A CORPUS that records no tokenizer is read as before. The speed with and without the corpus is not measured yet.
• docs/caches.md says why the first request after a restart can report a resident hit: the drain's conversation is read back at the start (#94).

Conductor and memory

• A model whose prompt has no place for the conductor's notes gets none, and its turns say so (SRV-703). The conductor puts its notes before the last message's <|im_end|>, and Flint's template ends no message with a token. On Flint every turn of a profile with memory on found no place: 1,456 turns in a day carried the error no_insertion_point, 199 of them with memory on, while the decision rows said memory: on. A family's fit has a notes key, Flint's is false, and such a turn records memory: {mode: off, why: model_notes}. Nothing went into Flint's prompt before either, so it writes the same text.
• Profiles have a fourth memory switch, none: nothing added, as an absent key meant. Every shipped profile sets its switch: on in chat, coding, hermes and chat-app, none in the others. A box's stored profiles keep what they hold.
• The Notes tab has a "Where memory is on" panel: the served model when it reads no note, and every profile with its switch and its clients.

Setup Guide and docs

• The Setup Guide sets the thinking effort (low, medium, xhigh, default medium) for every model, as it did for the two Qwen models (#100). docs/setup.md has a section on it, and the cards, docs/tool-eval-bench.md and docs/flint.md name the effort of each published number. Speed rows are greedy with thinking off. A fresh install thinks at medium, and the tool-eval-bench tools rows of the 27B and Ornith ran at xhigh.
• docs/server.md lists the timings and metrics fields, which chunk of a stream carries them, and the two draft formulas: tokens_per_block = (completion_tokens - 1) / blocks, also mean_acceptance_length, and draft_acceptance_rate = draft_n_accepted / draft_n (#99).
• The manual's ledger table closes the group its last paragraph opened, which held to the end of the book (#126).

Chat tab

• The page check and the browser drive close their Chrome in about 0.1 s (#131). The kit sent SIGTERM to Chrome's whole process group, so its helpers died at once and the browser process, waiting for them in its shutdown, sometimes stayed up until the SIGKILL 5 s later: on CI runners 3 closes in 200, and 17 in 200 with the CPUs busy. The SIGTERM now goes to the browser process alone, which ends its helpers, and to the rest of the group once it is gone. On the same runners 0 closes in 400 took more than 1 s.

Install and dashboard

• tandem-dash serve starts without the engine (#108). It exited with ModuleNotFoundError: No module named 'engine', as the standalone proxy ships proxy and server only. The conductor reads its knobs through server/knobs.py: the engine's registry where it is installed, else the environment with the same defaults.
• The dashboard's tests run on vitest 5.0.3, for two moderate advisories in @vitest/mocker 5.0.1 (#128). Nothing of it is in dashboard/dist.

Tests and CI

• CPU-job tests that failed at random are fixed at their cause (#113, #120, #122, OPS-231). The disk-tier tests wait for a staging slot, Flint's and LFM2's structure checks run in fp64, the page check bounds Chrome's start and not the whole check, and its test starts Chrome once before it times a start. The page check's Chrome picks its own debugging port. The preview's long-poll stops watching the client once its answer is complete, so a client that closes after the answer leaves the relay to end by itself. The proxy gate's tests hold a kept retry to the gate's own bounds.
• The dashboard's size budget measures with pako, zlib in JavaScript, so dist/ gets the same gzip size on every Node (#130). Node's own zlib depends on how Node is built: the same bytes read 470,034 with stock zlib and 471,208 on the official Linux builds, against a budget of 471,040, so npm test failed on Linux. The bundle sits 1,006 bytes under its 460 KB.
• tests/test_manual_braces.py checks that every table the manual's generator writes closes each brace it opens (#126).

Not in this release

These stay open for a later release:

• Speed at the card's sampling, a sampled row next to each greedy one (#95). It needs the GPU.
• The cards' slot count, and a check of two slots on Ornith (#101).
• The speed of the lookup with no corpus, the public corpus and the served one (#97). The corpus build itself is in this release.
• Flint fails to load when the corpus of another tokenizer is set (#105). It waits for its reporter's pull request.
• The dashboard's Docker build, which misses package directories (#87, #88). It waits for the contributor.

TandemLLM 0.14.1 2026-10-22 documentation

TandemLLM 0.14.1 fixes how Flint's tool calls reach a client and how it answers with thinking off, and moves model code behind adapters. A call's argument names are matched to the offered tool's schema, so Flint's read with path reaches opencode as filePath. A call to a tool the request did not offer goes to the offered tool of its kind. With thinking off, Flint answers without writing its reasoning into the answer. Flint can keep its full layers' keys and values in e4m3, off by default. Each model family now registers itself and declares how it is served, and shared code names no model. The other models write the same text as in 0.14.0 and keep their saved conversations. Flint's fingerprint moves once, so its saved prompt-cache entries from 0.14.0 are rebuilt once. Every change is in CHANGELOG.md, and the PDF below is the manual.

Tool calls

• Argument names are matched to the tool's schema before a call leaves the server (SRV-691), on both paths and in every tool-call form. A name is kept when the schema has it. Otherwise it is matched by case, _ and -, then by a small synonym table: a path for the schema's one path-like parameter, query / pattern / regex, cmd / command, content / text / contents. A rename happens only when exactly one parameter is the target and the call has not set it. A value is never changed, except to take its target's JSON type. A stream renames a key as soon as it is decided, so the streamed call and the parsed one agree. The journal records each rename (ARGS, RENAME). Through opencode's checks at Flint 4.0's card sampling, 8 runs an arm: calls the client refused fell from 4 to 0 under the conductor and from 2 to 0 raw, and the checks passed went from 28 and 29 of 32 to 32 of 32.
• A call to a tool the request did not offer (SRV-693) goes to the one offered tool of its kind when its arguments then name that tool's required parameters. A search or a listing with no such tool becomes a quoted grep or ls for the client's bash tool. A write, edit or delete never becomes a shell command. Anything else goes out as written, and the conductor's C4 asks for it again with the offered list.
• The conductor's call checks read the calls as the client receives them, after the renames (SRV-691).
• tools/client_smoke.py --sampling card runs the tool checks at the server's default sampling (SRV-692). The release gate stays greedy.

Flint

• With thinking off, Flint's prompt gets one line before the conversation: "Do not show your reasoning. Reply with the answer, or call a tool when one helps." (SRV-697). Without it, Flint opened its answer with reasoning on 7 of 10 hard asks greedy and 13 of 20 at its card's sampling, with a median of 582 and 600 tokens (600 was the cap). With the line it did so on 0 of 10 and 0 of 20, with a median of 153 and 116 tokens. On tool-eval-bench (4.0, raw path, thinking off, one run each) it scored 75 with the line and 77 without. Thinking on is unchanged.
• Flint's requests keep system and developer messages where they stand, with their own role, since its template renders them in place. The 27B's rule (one system prompt first) is now the default family setting, unchanged for the other models.
• FLINT_KV_FP8=1 keeps the 14 full layers' keys and values as e4m3 codes with one fp32 scale a (head, token): 14,784 bytes a token instead of 28,672 (SPD-150). This changes the output, so it is off by default. On 4.0 against the exact cache, the last 1,024 tokens of a 32,768-token source file read at KL 0.070, 93.8 % top-1 agreement and perplexity 7.465 against 7.165. At 122,880 tokens the numbers are KL 0.050, 95.7 % and 4.132 against 4.053. Plain decode runs at 79.4 tok/s against 60.6 at 64k, 57.1 against 39.3 at 128k and 36.6 against 23.05 at 256k. Prompt reading is 5 to 12 % slower. docs/flint.md has the table.
• To turn it on in the Setup Guide, pick Flint, go to the Tools step and switch on "Long-context cache (FP8)" under Features, then finish Review and Install. The switch shows the cost and the gain, and the memory bar counts the smaller cache.

• To turn it on from the command line, add QWEN38_FLINT_KV_FP8=1 at the end of ~/TandemLLM/run/ops/serve.env and run ~/TandemLLM/bin/tandem restart. /health then shows "kv_cache": "e4m3" under weights. To turn it off, delete the line and restart, or switch it off in the Setup Guide.

• tool-eval-bench on Flint's three profiles: bench 2.8.1.dev13 (commit f2e368b), seed 42, temperature 0, one run a row (n=1), the Setup Guide's Flint defaults (one slot, effort medium by default, FP8 cache off), BF16 at a 131,072-token window, 2026-10-22. Score, then the time the row took. Raw rows send X-Tandem: raw; bench-pk is a test profile we do not ship. Hard mode at n=1 varies a lot between runs, so we claim no ranking between the sizes. docs/flint.md has the chart and models/flint/measurements/2026-10-22-tool-eval-bench.json every row.

Model adapters

• Each family registers itself in its package's arch.py and keeps its own knobs in knobs.py (ENG-405). Adding a model edits no shared file.
• A family's ServeSpec says what the server needs: the tool-call form, how the model opens and turns off its reasoning, and where system messages may stand (ENG-406). The identity note names the model from the profile's card and has no default model.
• The tool-call forms (XML, Pythonic, arg-key) sit behind one ToolFormat interface in server/toolformats/, held to one contract by tests/test_toolformats.py (ENG-407). engine/grammar.py is back to its 0.13 bytes.
• A family declares which of its files others build on and which it uses from others, and kernel launch options come through EngineHooks (ENG-408). A change to a shared file moves exactly the declared dependents' fingerprints.
• Community-pack rules live in each base model's model.yaml, and its profile writer and pack check live in its family (ENG-409). The 27B's builder, drafter stack and side slots moved from server/app.py into engine/models/qwen3_5/serving.py (ENG-406, ENG-410).
• Guards hold this (ENG-404). They check v0.14.0's fingerprints as goldens with only Flint allowed to move, and they freeze SAME_ROWS. They also check that a change in one package moves only the adapters that hold it, and lint the shared modules for model names. docs/adding-a-model.md says what a family declares.

Tests and CI

• The CPU job on GitHub runs every test file, and a failure fails it (OPS-231). Timing tests bound CPU time against a reference measured in the same run, the two fp32 references bound their error in ulps, and the padded-row check bounds its difference by 2 % of the largest value, a quarter of what padded rows that move the state change. The shared-module lint reads f-strings the same on every Python. The slow-subscriber test gates its reader, and the drain test waits for its signal handler. The gate's fake engine runs in a process of its own. A test server binds before its port is read (--port 0 works), and helpers that need a port up front take one from a locked counter.

TandemLLM 0.14.0 2026-10-22 documentation

TandemLLM 0.14.0 adds Recursive's Flint, a mixture-of-experts model for coding agents, from our EXL3 packs at 4.0 and 6.0 bits a weight and from Recursive's BF16 checkpoint. Every change is in CHANGELOG.md, and the PDF below is the manual.

Flint

• 0xBakeer/TandemLLM-Flint-EXL3-4.0 at a04d4b95, 20.6 GB, and 0xBakeer/TandemLLM-Flint-EXL3-6.0 at 17e16040, 30.5 GB, public. Every projection is at 4 or 6 bits, the head at 5 or 8.
• exllamav3 1.6.2 converted them from the BF16 checkpoint, with a patch that adds the architecture; the patch is in each repository. The top layers' expert activations pass fp16's range, so the packs store each routed expert's up projection divided by 128 and say so in config.json.
• ops/use-profile.sh flint-exl3-4.0 downloads the default and starts it, and the Setup Guide lists all three installs: 4.0, 6.0 and flint-bf16 from RecursiveSI/flint. A download needs no token. The packs and the BF16 checkpoint are in one collection.
• The engine runs it on a new family, engine/models/flint: 35 layers with a 513-token window kept in a ring, 14 full layers without positional encoding, a learned sink per head, and 6 of 256 experts and a shared one a token. It serves two requests at once, with a window up to 262,144 tokens.
• Flint has no draft layer, so every profile drafts with the lookup, cut by StairCut from the pack's price file. After a round where StairCut drafted nothing, the engine takes 1, 2, 4, then 8 plain steps before it looks again.
• Flint opens its own reasoning block. Thinking off answers at once, and an effort level or a token budget closes the block at its cap.
• Flint writes tool calls in its own form. The conductor's checks read them in that form, a long string argument streams as the model writes it (SRV-687), and tool_choice required writes the calls in that form too (SRV-689).

Measured on one DGX Spark on 2026-10-22, a fresh server for each row, greedy, thinking off, 512 tokens, one run each. The last column is a 2,048-token answer that writes a whole HTML game:

Every drafted text is the plain run's. On prose the lookup runs 0.5 to 0.9 % under plain: it drafts 7 of 340 rounds, and those cost about what they save.

Long prompts slow the decode, because every token reads the 14 full layers' keys and values. The 4.0 pack decodes at 101.7 tok/s at 16k, 61.0 at 64k, 39.5 at 128k and 23.0 at 256k. A verify block reads them once, so the lookup gains more there: behind a 124,000-token prompt of code, a copy ask ran at 79.5 tok/s against 41.3 plain and an edit ask at 129.7 against 42.4. A 4,096-token prompt reads at 5,375 tok/s.

Against BF16, on 50 rows of 2,048 tokens of docs written after the model's training cutoff, the 4.0 pack gets KL 0.194 and top-1 79.7 %, the 6.0 pack KL 0.070 and 90.8 %. Flint is sensitive to rounding: two BF16 forwards of the same weights already differ by KL 0.034. Take 4.0 for speed and 6.0 when the text matters more. docs/flint.md has the checks, and measurements/2026-10-22-summary.json the runs. Sampled decoding, images and repeat runs were not measured.

Other changes

• A turn resumed from a session snapshot keeps its end anchor (SRV-678). The next turn had read 21,782 tokens again after a restore at 20,480.
• End alignment serves a tool call the template wrote back differently as the model wrote it (SRV-679), so the turn after it resumes at its end.
• Every prompt attention reads the KV heads as they are (SRV-673, SRV-681). At 131k one layer's NemotronH MTP step went from 28.5 to 1.6 ms, and a 179,678-token Ornith replay from 23.2 to 38.3 tok/s, with the same bits.
• After a lost CUDA context /health fails, a completion gets a 503, and the watchdog restarts the engine (SRV-671). A scratch turn can no longer forward an image's key ids into the embedding, which had caused it (SRV-669).
• In the Chat tab, screenshots fold in blocks, so a long run no longer reads every turn again (SRV-676), plan mode keeps the chat's effort word (SRV-663), and the run monitor steers a run that circles the same files, then stops it (SRV-677).
• When a tool call does not parse, the note to the model shows the call's form with a tool the request offers, in the model's own form, and no placeholder it could copy (SRV-688).
• tools/release.sh makes a release with one command (OPS-223), and tools/chat_fork.py runs the live model from a recorded step (SRV-667).

The other models write the same text as in 0.13.0 and keep their saved conversations.

Known issues

• At its default sampling (temperature 1.0), Flint 4.0 named a wrong argument or a tool the request did not offer in about 2 of 8 opencode asks (SRV-690).

Update

Run the installer again: bash ~/TandemLLM/src/install.sh --yes --ref v0.14.0, then tandem restart and tandem chat restart. On a box you installed with --setup, add --setup --force-setup and run ops/use-profile.sh with your profile after it. For Flint, run ops/use-profile.sh flint-exl3-4.0; it downloads 20.6 GB and needs 56 GiB of free memory to start.

TandemLLM 0.13.0 2026-10-21 documentation

TandemLLM 0.13.0 adds Ornith AI's Ornith-1.5-122B, an agentic and coding model on the Qwen3.5 mixture-of-experts layout, from our EXL3 pack at 4.0 bits a weight. It reads images, thinks before it answers and drafts with its own MTP layer. Every change is in CHANGELOG.md, and the PDF below is the manual.

Ornith-1.5-122B

• 0xBakeer/TandemLLM-Ornith-1.5-122B-EXL3-4.0 at 92f7483c, 64.6 GB, public. Every projection is at 4 bits, the head at 6, the MTP layer at 4 and the vision tower at 6.
• exllamav3 1.6.2 converted it from the BF16 checkpoint. Its 13 experts with all-zero fp16 weights needed a small patch to the quantizer, which is in the repository.
• ops/use-profile.sh ornith-1.5-122b-exl3-4.0 downloads it and starts it, and the Setup Guide lists it. A download needs no token. The base model, the pack and incoai's DFlash2 drafter for Ornith are in one collection.
• The engine runs it on a new trunk, engine/models/qwen3_5_moe: 36 Gated DeltaNet and 12 gated attention layers, 8 of 256 experts and a shared one a token, on the same EXL3 kernels as Flash-Next. It serves one request at a time, with a window up to 262,144 tokens.
• The profile drafts with the MTP layer and the lookup. After a round where StairCut drafted nothing, the engine takes 1, 2, 4, then 8 plain steps before it tries again. QSE_ORN_HEAD=dflash2 switches to the DFlash2 drafter.

Measured on one DGX Spark on 2026-10-20, a fresh server for each row, greedy, thinking off, 512 tokens, one run each. The last column is a 2,048-token answer that writes a whole HTML game:

Every drafted text is the plain run's. A start loads the pack in 65 s, and the weights take 61.16 GiB with the MTP layer.

Against the BF16 checkpoint's logits, on 8 wikitext rows of 1,024 tokens, the engine gets KL 0.139 and top-1 86.2 %. exllamav3's own forward of the pack gets KL 0.136 on the same rows. Against that forward on the same pack, the engine's top token agrees at 498 of 512 positions, so the two are close but not equal. Of the 14 differences, 9 have a margin under 0.5 logits and one a margin over 4. The cause is not tested.

measurements/0.13.0-summary.json has the runs. Requests at once, prompt reading speed, images and repeat runs were not measured.

On tool-eval-bench, one run a row on this release, Ornith scored:

docs/tool-eval-bench.md has every row, the misses and the flags.

Other changes

• The response cache keys on the effort level a request resolved to (SRV-645). A raw xhigh tool-eval-bench run had replayed 190 answers of the high run before it.
• tools/teb_batch.sh and tools/teb_summary.py run tool-eval-bench rows one at a time and keep what each row ran under (OPS-209).
• A package whose result an offered tool delivers can stay out of the turn (SRV-615). It ships in shadow: it logs what it would keep out and changes no prompt.
• In the Chat tab, grep given one file's path searches that file (SRV-649), and read_file reaches every line of a file over 4 MiB (SRV-651).
• tests/test_grammar.py passes every run (SRV-647).
• Ornith's caches have room (SRV-655). At the first ceiling of 74 GiB every end anchor and store snapshot was refused; the limits are now 86 and 90 GiB with a 2 GiB pin, and a store decline names the memory policy and its figures.
• When the end alignment fails on a rewritten tool call, the engine serves the part it aligned and resumes there (SRV-657). On Ornith one such turn had read 132,635 of 144,923 tokens again.
• In the Chat tab, writes and long commands stay whole in later requests until a fold point (SRV-659), and plan mode and a tool check that times out keep the request's head (SRV-663). A run stopped at its 100-call cap leaves the chat idle, and render_page keeps a shader error's compiler lines (SRV-665).
• Every test file runs its tests as a script, and a file that runs none fails (SRV-661). tools/chat_trace.py replays a recorded chat through chatd in seconds, and tools/prefix_check.py reads what each request reads again (SRV-667).

The other models write the same text as in 0.12.2 and keep their saved conversations. The router kernel now takes up to 512 experts, which moved the fingerprints of Flash-Next, LFM2.5 and NemotronH; at 512 it runs as before, and their first start adopts 0.12.2's files.

Update

Run the installer again: bash ~/TandemLLM/src/install.sh --yes --ref v0.13.0, then tandem restart and tandem chat restart. On a box you installed with --setup, add --setup --force-setup and run ops/use-profile.sh with your profile after it. For Ornith, run ops/use-profile.sh ornith-1.5-122b-exl3-4.0; it downloads 64.6 GB and needs 93 GiB of free memory to start.

TandemLLM 0.12.2 2026-10-20 documentation

TandemLLM 0.12.2 fixes two Chat-tab bugs from the issue tracker and brings back LFM2.5's older saved conversations. No model, card or profile changed, and every profile writes the same text as in 0.12.1. Every change is in CHANGELOG.md, and the PDF below is the manual.

Large reads in the Chat tab

A run that read a large file could die with a request over the window. AgentSparko reported it in #18. That request's estimate was 4,612 tokens low, and max_tokens left only 256 for the error (SRV-639).

• One step's tool results now share the room the model has left: half of what is left under the compaction line, 2,000 tokens at least, split across the step's calls.
• read_file stops at whole lines and ends with the offset to read on from. A line longer than one read goes on with a new column argument. A 700 KB file used to come back as its first and last 16,000 characters, with about 400 KB left out unsaid.
• Each request's estimate keeps a margin for text the engine has not counted yet. Near the window, chatd counts the request on our engine with POST /v1/tokenize and refuses one that is over before it sends it.

The #18 repro on the DGX Spark (Qwen3.8-27B, 262,144 tokens, a 700 KB .md, "read it and summarize") now ends with a summary.

read_file's schema changes, so each chat's first request after the update reads its prompt again.

The port preview streams

joeynyc found that the port preview held an app's whole answer until the app closed the connection (#86). A preview with Server-Sent Events or a long-poll never showed an event, and the sandbox counted as busy, out of reach of the idle stop (SRV-633).

• The answer goes on to the browser once its head is read. A page, script or style sheet up to 8 MiB is still read whole and rewritten. Bigger ones pass as they are, with X-Tandem-Rewrite: skipped.
• The app has 300 s to start its answer, else 504. If the client leaves, or takes no bytes for 60 s, chatd ends the relay inside the sandbox and frees it.
• The engine's chat forward ends its request to chatd when the browser leaves, also while chatd has nothing to send yet. On the DGX Spark a held long-poll ended in the sandbox within 0.2 s.

LFM2.5 saved conversations

0.11.2 changed two LFM2.5 modules without changing a stored row and gave them no equivalence entry, so 0.11.2 to 0.12.1 adopted none of the older tier files (ENG-381). On the DGX Spark, tools/lfm_attn_rows_check.py found the split attention equal to 0.11.1's byte for byte at 1 to 32 rows. The entries are in now.

Boxes that upgrade from 0.9.x to 0.11.1 adopt their LFM2.5 conversations again. Those that ran 0.11.2 to 0.12.1 already lost them.

Other fixes

• The dashboard's build uses source-map-js 1.2.2 (OPS-205, GHSA-68fv-2mgg-jv7q). It is a build-time dependency, and the served bundle is unchanged.
• Three CPU tests that failed on 0.12.0 and 0.12.1 pass (SRV-635, SRV-637, ENG-381). tests/test_security_docs.py checks SECURITY.md and the threat model against the tree (OPS-202).

Speed was not measured for 0.12.2, since no engine code that writes text changed after 0.12.1.

Update

Run the installer again: bash ~/TandemLLM/src/install.sh --yes --ref v0.12.2, then tandem restart and tandem chat restart. On a box you installed with --setup, add --setup --force-setup and run ops/use-profile.sh with your profile after it.

TandemLLM 0.12.1 2026-10-20 documentation

TandemLLM 0.12.1 serves the first community pack on Nemotron 3.5 Super VL: styles01's EXL3 pack at 3.5 bits a weight, text only, with LiLiCorr and the lookup. A request with images now takes a side slot, and eleven pull requests from three contributors fix chatd, the Setup Guide, the GGUF reader and the image path. Every change is in CHANGELOG.md, and the PDF below is the manual.

Super VL community pack

• aitamedia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-EXL3-3.5bpw at 358667cf, 57.8 GB, by styles01 (#63). Routed experts are at 3.5 bits, every other projection at 5, and the head is fp16. It holds no image tower, so it serves text.
• ops/use-profile.sh community/nemotron-super-vl-exl3-3.5bpw downloads it and starts it with the limits, sampler, LiLiCorr drafter and lookup of our nemotron-super-vl-exl3-4.0 profile. It shows in the Setup Guide under Community weights.
• Super VL is now a community base (ENG-377). python -m server.setup.community check-pack community/<id> checks a pack's files and names what differs.
• Its config ties the head to the embedding, but the pack stores its own head. NemotronH now reads a stored head in that case and notes it in the load audit.

Measured on one DGX Spark on 2026-10-20, one request at a time, 512 greedy tokens, thinking off, one run each:

Every drafted text is the plain run's. A cold prompt of 7,507 tokens reads at 529 tok/s. At load the weights take 53.58 GiB, and the served profile (262,144 tokens, 3 slots) starts and answers in 110 s. Against exllamav3 1.6.0's own forward on the same pack, the top token agrees at 510 of 512 positions; both others are near ties, and the KL is 0.0019.

Plain decoding runs at 25.9 tok/s on this pack and at 31.1 on our 4.0 pack. From the headers, a decode token reads 7.95 GB of this pack and 6.74 GB of ours. measurements/0.12.1-summary.json has the runs. Images, requests at once and repeat runs were not measured.

Images on a side slot

A request with images now takes a side slot the way a text request does (SRV-629). On 2026-10-19 two Chat-tab agent runs sent all 31 of their requests, each with 3 to 8 screenshots, to slot 0 while slot 1 sat idle: 56.6 minutes of wall time for 18.9 of decoding. Each side slot keeps its own image context and rotary offset, and on the 27B one image request wrote the same 300 tokens on slot 1 as twice on slot 0. QWEN38_SIDE_IMAGES=off keeps images on slot 0, as in 0.12.0.

Fixes

• The Chat tab's port preview keeps an app's redirects inside the preview (SRV-631). The Chat view starts nothing when it closes during a sign-in (VIS-235).
• joeynyc, #78 to #85: chatd sends a text attachment it could not save, measures the inline budget on the branch a message goes on, and forwards the port preview's path as sent. The Setup Guide's remove checks for a running job under the lock. GGUF headers with sizes that overflow are refused, and a 16 by 40,000 pixel image no longer fails with a 400.
• AgentSparko, #34 and #43: a Finish answer that lands after you left the Setup Guide opens no install page (issues #4 and #5). chatd's sandbox reader closes its stream on every exit, and an MCP server's body and SSE lines are capped.
• SECURITY.md says how to report a vulnerability in private.

Every profile writes the same text as in 0.12.0. The NemotronH fingerprint changes, so saved Bolt, Lightning and Super VL conversations from 0.12.0 read their prompt again on the first request.

Update

Run the installer again: bash ~/TandemLLM/src/install.sh --yes --ref v0.12.1, then tandem chat restart. On a box you installed with --setup, add --setup --force-setup and run ops/use-profile.sh with your profile after it.

TandemLLM 0.12.0 2026-10-19 documentation

TandemLLM 0.12.0 gives API clients that offer tools a conductor profile of their own. An OpenAI SDK, curl or an unnamed client that sends tools and names no profile now gets tools: the prompt as the client sent it, token for token the raw path's, with the loop guards and the tool-call checks acting. Every change is in CHANGELOG.md, and the PDF below is the manual.

Tool calls

• The tools profile. A request resolves to the X-Tandem-Profile header, else the profile mapped to its client, else tools when no profile maps the client and the request offers a function tool. It adds no note, package, memory or effort change. D1, D1b, D2, E1, C3, C4 and C5 act, and each act is journaled. A call goes out whole once the checks have seen it.
• Structured outputs with tools. A response_format or structured_outputs beside tools was a 400. The answer is now a tool call or the document, and the model's own first answer token picks which. Speculation changes no token.
• C5, one mutation per turn. An answer with two or more calls that change something outside the conversation (send, schedule, create, write, delete) is written again one call at a time, each after the last one's result. It acts in tools and journals in shadow in coding, chat-app, hermes and chat.
• The secret echo mask. A key that a tool result showed the model is masked where the answer repeats it: sk-abc123 becomes sk-[redacted]. Call arguments are untouched.
• On the raw path an SDK, curl or unnamed client gets the thinking in reasoning_content and the answer alone in content.

tool-eval-bench

We ran tool-eval-bench on one DGX Spark with Qwen3.8-27B in the EXL3 6.0 pack, seed 42, temperature 0, each row one run. On the 69 scenarios:

Two runs of the same prompt scored 95 and 97, so rows within 2 points are the same. In the bench's hard mode, 28 scenarios, the conductor scored 91 and raw 84, one run a side. docs/tool-eval-bench.md has every row with its settings and command, and docs/benchmarks.md says how to run the bench.

Fixes

• /v1/models/{id} asks for the key /v1/models asks for. The standalone proxy times out a request that sends nothing for 60 s. chatd compacts and retries once on a 400 that says the request is over the window. An operator sets a relayed model's output limit under Backends (#21).
• Pull requests #36, #46, #47, #49, #50, #51, #53 to #61, #64, #67, #70, #71 and #73 to #76 by AgentSparko fix the conductor's journal writer and session pages, the collectors, the proxy's TLS sockets and model cards, the sandbox relay, chatd's finish_guard, and stale responses in the dashboard.

Update

Run the installer again: bash ~/TandemLLM/src/install.sh --yes --ref v0.12.0. A box with stored conductor documents gets the tools profile at its next start; a stored monitor document gets C5 with tools/monitor_migrate.py --put http://127.0.0.1:8000 --tasks C5, run on the box.

TandemLLM 0.11.4 2026-10-18 documentation

TandemLLM 0.11.4 adds Agents mode to the Chat tab. The main agent plans with you and hands every file write and every multi-step build or fix to a worker, so the file texts live in the worker's context and leave with it. The engine keeps the worker's and the parent's cache rows while they take turns: in the live checks, no worker call after the first read its prompt from 0. Every change is in CHANGELOG.md, and the PDF below is the manual.

Agents mode

• A chat setting, agents, off by default. The composer's agents chip turns it on for one chat, and the settings switch sets the default for new chats. In a chat with messages the switch first says how many tokens the next message reads again.
• The main agent gets a task tool. Each call starts a worker: a full run on a hidden conversation that sees the task, its brief, the todo list and your message, and nothing else of the chat. The worker uses the chat's sandbox and files and makes at most 40 calls; a run starts 8 tasks at most.
• In Agents mode the main agent reads and delegates. It has read_file, glob, grep, web search, render_page and task, and bash, write_file and edit_file tell it to give the work to a worker.
• chatd checks the brief and the report. A brief needs its Goal, Files, Constraints and Check lines and carries no code. A report that misses a field goes back to the worker once. Below each report chatd lists the files the task changed, the files the report left out, the checks that ran and the claims with no call behind them.
• A Task card shows each worker: its status in words, call k of max, tokens, time, the brief, the report, chatd's check, its screenshots and its transcript. Stop, a queued message and Continue reach the worker.
• The Context panel shows each running worker in its own section, with its context bar, its prefill bar and tok/s, or its decode rate, first token and acceptance. The Now dock has a row per worker.

Chat tab

• A written file's text goes to the model as a stub from the next request on (path, lines, bytes, sha256), and a long read or command output stays whole until a compaction. The store keeps every text. One run on 2026-10-17 had grown from 25k to 162k tokens in 50 minutes.
• A compaction cuts until the request lands under half the limit that set it off. The log gives the size before and after.
• A memory refusal compacts the chat and the run goes on, instead of retrying the same prompt for 10 minutes and ending interrupted.
• A run on a cloud model takes no run slot, so a DeepSeek chat no longer waits behind two chats on the engine.
• Tool screenshots are append-only, so an earlier screenshot no longer changes the prompt. render_page says the check browser has WebGL2 and no WebGPU.
• A failed finish_run no longer wedges a conversation (#41), and chatd's store releases its lock when a transaction cannot begin (#14).

Engine and caches

• A worker's request (X-Tandem-Worker, X-Tandem-Parent-Session) is never a guest of another conversation. For 600 s after a worker's last request its parent's rows go to the NVMe rather than away, and a returning parent tries slot 0 first.
• The park and the NVMe spill keep conversations they used to drop: a swap parks the longer conversation whole, a conversation the park refuses goes to the NVMe, and a restore parks the conversation it reads over.
• A request with images resumes at the end anchor, as a text request already did (QWEN38_MM_END=off restores the old path).
• The live row shows a prefill's rate from its first progress mark and while a layer-major window runs.
• These restores copy rows the engine computed before and compute none. The tests hold every such resume's logits to a cold prefill's within 1e-4; the served text against 0.11.3 was not measured.

Proxy and setup

• The proxy closes a socket whose TLS handshake failed, which leaked one fd per failed request (#38).
• The Setup Guide's reset and an apply cannot run at once (#40).

Pull requests #14, #38, #40 and #41 are by NotMainstream.

Upgrade

Run the 0.11.4 installer again, then restart the engine and chatd. chatd's store moves to v8 in place at its first start. On a box you installed with --setup:

curl -fsSLO https://raw.githubusercontent.com/0xBakeer/TandemLLM/v0.11.4/install.sh
bash install.sh --setup --force-setup --yes
bash ~/TandemLLM/run/ops/use-profile.sh <your profile>
tandem chat restart

A box installed without --setup upgrades with bash install.sh --yes and the same --dir, then tandem restart and tandem chat restart.

TandemLLM 0.11.3 2026-10-17 documentation

TandemLLM 0.11.3 makes an install through the Setup Guide safe to retry: it checks the box before anything stops, repairs what it can, and says in plain words what broke. Super VL reads images, and the Chat tab measures a cloud model's turns and prices them. Requests on the raw path write the same text as in 0.11.2. Every change is in CHANGELOG.md, and the PDF below is the manual.

Setup Guide

• tandem doctor checks an install and repairs what is safe: file modes, the run/ops copies, stale pid files, a hold left behind, the secrets file's mode. It reports what it does not change, such as a serve.env that does not load. It runs before every tandem start and in every Setup Guide install, and as tandem doctor [--check] [--json].
• A new step checks the new setup before the restart: the scripts, serve.env with the new profile, the weights, the memory, the port, the interpreters. A failure ends the job there and nothing is stopped or changed.
• TandemLLM's scripts call each other through bash, never through the execute bit. A start script that lost its execute bit no longer fails the restart.
• A failed step says what broke, shows what tandem doctor repaired, and offers Retry from that step and Copy diagnostics.
• A restart is one card with the job's step, the load's phases and the time left. A refresh or an offline page goes back to the job. A finished install has its own screen with Start chatting.
• The status responder belongs to its tree: a responder of another checkout or an older release is replaced, and it carries the job and GET /v1/setup/status while the engine is away.
• The weights step lists the weights on the box and removes a model's weights. It never removes the running profile or a folder another profile shares, and it writes the line that downloads them again.
• tools/release_gate_setup.sh installs a candidate twice in scratch folders beside the live engine and runs the whole flow in a phone-sized browser, once with the execute bits stripped.

Chat tab and cloud models

• Nemotron 3.5 Super VL reads images through its own vision tower: uploads and the screenshots of render_page.
• A cloud model's turn has its tok/s, the first token's wait, its thinking tokens, the provider's cache hits and its cost. Performance and Usage keep the engine's requests apart from relayed ones, and Usage has a table of the engine against each backend.
• The picker takes names, limits, image input and effort levels from the provider's model list. Images go only to a model that reads them, and a model id that appears later is stored disabled until it is turned on under Backends.
• A run that builds or changes a web page checks it in a browser before it ends and closes with what was checked and what was not.
• render_page reports the page's own console, failed loads and errors with the line of the chat's file. A run may make 100 model calls, and a chat sandbox has 4 GiB of memory.

Engine and conductor

• A conversation's next turn goes to the side slot that holds the longest share of its prompt, so it no longer reads the conversation again on another slot.
• A turn that ends inside a verified block commits through its end token, so NemotronH models keep their end anchor after tool calls.
• The narrator is off by default, and the decider's memory claim is 3.0 GiB, so Laya fits beside Super VL.
• Under the conductor a turn can get packages it did not get before (the class cores), and its text changes with them.

Upgrade

Run the 0.11.3 installer again, then restart the engine and chatd. The conductor's migrations run at the engine's first start, and the usage ledger moves to schema 10 in place. On a box you installed with --setup:

curl -fsSLO https://raw.githubusercontent.com/0xBakeer/TandemLLM/v0.11.3/install.sh
bash install.sh --setup --force-setup --yes
bash ~/TandemLLM/run/ops/use-profile.sh <your profile>
tandem chat restart

A box installed without --setup upgrades with bash install.sh --yes and the same --dir, then tandem restart and tandem chat restart.

TandemLLM 0.11.2 2026-10-16 documentation

TandemLLM 0.11.2 makes the Chat tab work on a fresh install. chatd, the Chat tab's server, now starts and stops with the engine, and the Chat tab says when chatd is down. Super VL gets its full 1,048,576-token window. Update every box that runs the Chat tab. Every model writes the same text as in 0.11.1. Every change is in CHANGELOG.md, and the PDF below is the manual.

chatd with the engine

On a plain or --setup install nothing started chatd, and the Chat tab fell back to an in-browser mock without saying so. Aron reported it in issue #3. Now tandem start, ops/start.sh, the systemd unit and the Setup Guide start chatd with the engine, and tandem stop stops it. An engine restart leaves it running, and the watchdog starts a chatd that died. tandem chat start|stop|restart|status|logs drive it, tandem status has a chatd line, and QSE_CHATD=off opts out.

The Chat tab names the problem when chatd is not running, with the command to run, instead of loading the mock. The mock now loads only for ?chat=mock.

Setup Guide and Super VL

The Setup Guide sets the number of chats and both windows with steppers. A model card's range is how far the model was measured, not a cap: above it the step says the count is untested, and the memory estimate decides, up to the engine's 16 chats. A window walks 8,192, 16,384 and on up to the model's own maximum, and the field takes any number, such as 128k or 1m.

Nemotron 3.5 Super VL takes a window up to 1,048,576 tokens, its checkpoint's own, for slot 0 and the side slots. I checked it to 249,983 prompt tokens, five needles of five found. A 1,000,000-token prompt is not measured yet.

Install fixes

• bin/tandem loads its config on an install with no ops/active-profile. Before, tandem start and status ended with status 1 and no message on every plain and --setup install.
• The installer copies roles.sh, watchdog.sh and hold.sh into run/ops. Every start printed roles.sh: No such file or directory.
• ops/chatd.sh runs under the install's venv and reads the secrets file the engine reads. ops/make-secrets.sh writes that file, so the Setup Guide's generated admin password takes effect.
• The Setup Guide's reset keeps an install's state/secrets.env.

Chat tab

• Each chatd labels its sandbox containers and touches only its own. Before, a second chatd on the same Docker adopted the first one's containers and removed them.
• A sandbox container removed or stopped outside chatd starts again at the next tool call. The same sandbox failure three calls in a row ends the run, and Continue goes on once the sandbox is back. The Continue card says why a run stopped.
• The preview pane opens the running app. A port that starts listening opens in a Preview tab, a waiting pane names the command and offers Start, and present reports what the page loaded. render_page and the preview dial ::1 and 127.0.0.1 by address.
• The model picker lists models, one row each. DeepSeek V4 Flash and V4 Pro through the proxy can be picked, and a DeepSeek run no longer fails on its output limit.
• The Context panel shows a live run's figures and says why one is missing. The More menu opens downward. The Chat tab has no horizontal overflow from 320 to 1440 px.

Upgrade

Run the 0.11.2 installer again. It rewrites bin/tandem, run/ops/serve.env and the scripts in run/ops, and chatd then starts with the engine. A chatd started earlier keeps the old code until tandem chat restart. On a box you installed with --setup:

curl -fsSLO https://raw.githubusercontent.com/0xBakeer/TandemLLM/v0.11.2/install.sh
bash install.sh --setup --force-setup --yes
tandem chat restart

A box installed without --setup upgrades with bash install.sh --yes and the same --dir.

TandemLLM 0.11.1 2026-10-16 documentation

TandemLLM 0.11.1 is a security fix for the Chat tab. Update every box that runs it. The release also gives Bolt's EXL3 packs from 2.5 to 6.0 bits their StairCut price files, so drafting on them is no longer slower than plain decoding. Every model writes the same text as in 0.11.0. Every change is in CHANGELOG.md, and the PDF below is the manual.

Security

chatd, the Chat tab's server, reads and writes two folders that a chat's sandbox can change: the chat's workspace and the sandbox's home. In 0.10.0 and 0.11.0 chatd followed a symlink there. Code running in the sandbox could point an attachment upload, a file tool, a snapshot or the Open WebUI runner's files at a path outside the sandbox. chatd then read or wrote that path with the rights of the user it runs as.

In 0.11.1 no access in those folders follows a symlink. chatd opens every folder on the way with O_NOFOLLOW, writes a new file and renames it into place, and reads regular files only. A path with a symlink on it is refused. An attachment that cannot be saved under uploads/ still goes out with its message, and the prompt says why. Joey reported the upload case.

Upgrade

Run the 0.11.1 installer again, then restart chatd. chatd is its own process, so it keeps the old code until it restarts. On a box you installed with --setup:

curl -fsSLO https://raw.githubusercontent.com/0xBakeer/TandemLLM/v0.11.1/install.sh
bash install.sh --setup --force-setup --yes
bash ~/TandemLLM/src/ops/chatd.sh stop
bash ~/TandemLLM/src/ops/chatd.sh start

A box installed without --setup upgrades with bash install.sh --yes and the same --dir.

Bolt EXL3: drafting no longer loses to plain

In 0.11.0, Bolt's EXL3 packs at 3.0, 3.5 and 4.0 bits wrote prose and chat slower with LiLiCorr than with plain decoding: prose at 31 to 33 tok/s against about 41 plain. The packs from 2.5 to 6.0 had no StairCut price file. Without one, the cut kept a price of 1 ms for a one-row block, so it often threw away a draft it had already paid for. Each pack now ships its price file, measured on the Spark.

One DGX Spark, 2026-10-16, tok/s. One request through the server, 512 greedy tokens, thinking off, the mean of 3 runs. The drafted columns use LiLiCorr and the lookup, and every drafted text equals the plain run's. These rows are now in the setup guide's speed table.

Also in 0.11.1

• Two contributions by joeynyc. A pinned asset now downloads at its revision (pull request #1). NemotronH's saved-state fingerprint now covers its GGUF reader and the GGUF file (pull request #2). Because that fingerprint changed, Bolt's and Super VL's saved conversations from 0.10.0 and 0.11.0 are not reused, and their first request reads the prompt again.
• Each weight card in the setup guide names its maker and its Hugging Face repository. turboderp's Flash-Next packs are listed as community packs.
• The install is two commands, then the setup guide in the browser at http://127.0.0.1:8000/dashboard/#/setup.

Known limits

• Super VL's EXL3 2.0 pack sometimes drafts prose below plain. In one of three runs on 0.11.1 it wrote prose at 33.5 tok/s against 36.5 plain; the other two ran at 37.3. The cause is not isolated yet.
• The other limits of 0.11.0 still hold: see its release notes.

TandemLLM 0.11.0 2026-10-16 documentation

TandemLLM 0.11.0 adds a fifth model, NVIDIA's Nemotron 3.5 Super VL 120B-A12B. It runs from our EXL3 packs at 4.0 and 2.0 bits a weight or from NVIDIA's own NVFP4 release, and every profile drafts with NVIDIA's LiLiCorr drafter for Super VL. The Qwen, LFM2.5 and Bolt models write the same text at the same speed as in 0.10.0, and their saved chats carry over. Every change is in CHANGELOG.md, and the PDF below is the manual.

Nemotron 3.5 Super VL 120B-A12B

Super VL has Bolt's hybrid layout at a larger size: 40 Mamba2 layers, 40 mixture-of-experts layers and 8 attention layers, with 22 of 512 experts per token. The engine now reads NVIDIA's ModelOpt checkpoints as stored, so NVIDIA's NVFP4 release runs without a conversion: NVFP4 experts, FP8 Mamba2 and attention projections and an FP8 head.

One DGX Spark, 2026-10-16. Quality: 8 wikitext-2 rows of 1,024 tokens against the logits of NVIDIA's BF16 checkpoint. Decode: one request through the server, 512 greedy tokens, the mean of 3 runs.

Use the 4.0 pack. Its KL is 0.008 above NVIDIA's NVFP4 release, and it decodes 1.6 times as fast. With LiLiCorr and the lookup it writes 53.6 tok/s on code, and edit and copy asks reach 101.0 and 108.3 tok/s. The 2.0 pack is 17 % faster plain, but its KL is more than six times the 4.0 pack's. Take it only when 4.0 does not fit.

Both packs are public repositories on Hugging Face, 0xBakeer/TandemLLM-Nemotron-3.5-Super-VL-120B-A12B-EXL3-<bpw>, pinned by commit, and both are in one collection. The NVFP4 release and the drafter come from NVIDIA's repositories. docs/nemotron-h.md has the comparison with vLLM on the same NVFP4 weights.

Upgrade

Run the 0.11.0 installer again, then switch to Super VL. On a box you installed with --setup:

curl -fsSLO https://raw.githubusercontent.com/0xBakeer/TandemLLM/v0.11.0/install.sh
bash install.sh --setup --force-setup --yes
~/TandemLLM/run/ops/use-profile.sh nemotron-super-vl-exl3-4.0 --dry-run   # each file's URL and size
~/TandemLLM/run/ops/use-profile.sh nemotron-super-vl-exl3-4.0

A box installed without --setup upgrades with bash install.sh --yes and the same --dir. A start of the 4.0 profile needs 90 GiB of free memory.

Known limits

• Super VL is text only. Its vision tower is not served yet, and neither is its MTP head.
• TandemLLM reads a prompt at 476 to 589 tok/s. vLLM reads the same prompts 4 to 5 times as fast.
• With a drafter, requests at once take turns. Three at once run at 19.6 tok/s in total on the 4.0 pack.

TandemLLM 0.10.0 2026-10-16 documentation

TandemLLM 0.10.0 adds a fourth model, NVIDIA's Nemotron 3.5 Bolt 74B-A9B. It runs from our EXL3 packs at 2.0 to 6.0 bits a weight or from NVIDIA's own 2-bit GGUF, and the EXL3 profiles draft with NVIDIA's LiLiCorr drafter. The Qwen and LFM2.5 models write the same text at the same speed as in 0.9.2, and their saved chats carry over. Every change is in CHANGELOG.md, and the PDF below is the manual.

Nemotron 3.5 Bolt 74B-A9B

Bolt is a hybrid of 40 Mamba2 layers, 40 mixture-of-experts layers and 8 attention layers, with 9 of up to 496 experts per token. The engine runs a verify row with the same bits as the decode step, so a drafted answer is the plain answer. Where a reasoning budget forces a thinking block closed, the close can land a few tokens later (docs/exactness.md).

The 2.5, 3.5 and 5.0 packs have profiles too. One DGX Spark, 2026-10-15 and 16. Quality: 8 wikitext-2 rows of 1,024 tokens against the logits of NVIDIA's BF16 checkpoint. Decode: one request through the server, 512 greedy tokens, the mean of 3 runs.

The 2.0 pack keeps its routed experts at 2 bits and its dense projections at 4. It scores better and runs faster than NVIDIA's GGUF. With LiLiCorr and the lookup it writes 65.7 tok/s on prose and 102.8 on code. Edit and copy asks reach 180.5 and 244.7 tok/s (512 greedy tokens, thinking off, 3 runs). On the 4.0 pack LiLiCorr gives no speed in this release: StairCut keeps plain steps, and we have not found why. Three requests at once run at 72.1 tok/s in total.

Each pack is a public repository on Hugging Face, 0xBakeer/TandemLLM-Nemotron-3.5-Bolt-74B-A9B-EXL3-<bpw>, pinned by commit, and all seven are in one collection. The GGUF and the drafter come from NVIDIA's repositories. docs/nemotron-h.md has the checks against transformers, llama.cpp and BF16.

Chat tab

• Each chat's sandbox has 1 GiB of memory, and a call that hits the limit fails with "out of memory".
• Attachments are never inlined past the window. A file over a quarter of the free window goes in as one line with its path under /workspace/uploads/.
• A tool call no longer waits without end for its sandbox. After 60 s it fails with the reason.

Upgrade

Run the 0.10.0 installer again, then switch to Bolt. On a box you installed with --setup:

curl -fsSLO https://raw.githubusercontent.com/0xBakeer/TandemLLM/v0.10.0/install.sh
bash install.sh --setup --force-setup --yes
~/TandemLLM/run/ops/use-profile.sh nemotron-bolt-exl3-2.0 --dry-run   # each file's URL and size
~/TandemLLM/run/ops/use-profile.sh nemotron-bolt-exl3-2.0

A box installed without --setup upgrades with bash install.sh --yes and the same --dir.

Known limits

• Bolt is text only. Its image tower is not served yet.
• The IQ2 profile runs without a CUDA kernel for its shared experts, so it decodes at 27.9 tok/s, not 32.7.
• The IQ2 profile drafts with the lookup alone. With LiLiCorr beside it, StairCut kept plain steps on this build, and the cause is not known yet. Edit and copy asks still run at 97.9 and 131.2 tok/s.
• The 2.5 to 6.0 packs have no StairCut price file yet. The cut learns their prices while it serves.

TandemLLM 0.9.2 2026-10-15 documentation

TandemLLM 0.9.2 fixes two install bugs that an early tester hit on a new DGX Spark with 0.9.1. The engine, the conductor and the dashboard do not change. Every change is in CHANGELOG.md, and the PDF below is the manual.

Fixes

• Checksums: fetch_weights.py compared a download's file globs to file names as plain text. Qwen3.5-0.8B (the shadow reader) and Qwen3.5-2B (the narrator) list only globs, so their download checked no file and stopped with 0 files checked: FAILED. Laya's files under multilingual/ and typed-decisions/ were never checked. Now the globs pick the files to check.
• EXL3 after install.sh --setup: the 27B's EXL3 profiles refused to switch, with base-plain already on disk. The installer wrote QWEN38_MODEL, an empty Hugging Face cache folder on a new box, after the weight profile, so the profile could not replace it. The installer's keys now come before the profile, and an EXL3 profile reads base-plain when that folder has no FP8 snapshot. A standalone community pack now reads its own folder on an installed box too.
• HF_HOME: an install with HF_HOME or HF_HUB_CACHE set writes that cache to run/ops/serve.env. A later shell without the variable fetches the base into the same cache and finds it there.

Upgrade

Run the 0.9.2 installer again. It rewrites run/ops/serve.env and the scripts beside it; a git checkout of the tag keeps the old ones. On a box you installed with --setup:

curl -fsSLO https://raw.githubusercontent.com/0xBakeer/TandemLLM/v0.9.2/install.sh
bash install.sh --setup --force-setup --yes
~/TandemLLM/run/ops/use-profile.sh <profile>

A box installed without --setup upgrades with bash install.sh --yes and the same --dir. On a new DGX Spark:

curl -fsSLO https://raw.githubusercontent.com/0xBakeer/TandemLLM/v0.9.2/install.sh
bash install.sh --setup --yes
~/TandemLLM/run/ops/use-profile.sh exl3-4.5 --dry-run   # each file's URL and size
~/TandemLLM/run/ops/use-profile.sh exl3-4.5

Known limits

The limits of 0.9.1 still hold: see its release notes.

TandemLLM 0.9.1 2026-10-14 documentation

TandemLLM 0.9.1 adds web search and media playback to the Chat tab, and turns on the loop guard and memory notes in the shipped conductor profiles. The decode path does not change. Every change is in CHANGELOG.md, and the PDF below is the manual.

What's new

• Web search in the Chat tab: a built-in web_search tool with DuckDuckGo (the default, no setup) or your own SearXNG instance, set in the chat settings. The model cites the URLs and reads a page with web_fetch.
• Media in the Chat tab: video, audio and PDF files open in a player or the browser's PDF viewer, in the Files pane and the present card. chatd streams a workspace file with byte ranges, with no 2 MiB cap.
• The loop guard is on in the shipped coding, chat-app, hermes, overnight and chat profiles. When the model makes the same tool call a third time in one user turn and the results were the same each time, the guard stops the call and the model writes a different one.
• The loop escape: after a repeated or near-repeated call, the session samples with the model card's settings until it makes progress, then decodes greedy again. A client's own sampling wins. Output changes on those turns only.
• Memory notes are on in the shipped chat, coding, hermes and chat-app profiles, so the Chat tab sees them too. The model is told the notes are its memory between chats. The Context panel shows a turn's notes.
• --reasoning-effort none and off turn thinking off for every request that says nothing about thinking.

Fixes

• Setup Guide: after a reload, a finished install job stays finished and keeps its whole log. The change list names the keys a model switch drops from run/setup.env.
• Setup Guide: the fetch step downloads the models of the tools you turn on (the shadow reader, the narrator, Laya). A failed shadow reader load is tried again after 300 s.
• The 27B's EXL3 profiles fetch only the base tensors an EXL3 load reads, by HTTP range requests: 3.54 GB instead of 30.41. A fresh exl3-6.0 downloads 36.48 GB, not 63.34.
• Behind a reverse proxy, a 502, 503 or 504 during an engine restart shows the dashboard's cached page, which reloads once /health answers.

Upgrade

Stored conductor profiles migrate once at the first start. In the five profiles above, the loop guard turns on unless a stored version of the profile ever had it on, and a profile without a sampling section gets one with the loop escape on. A stored chat, coding, hermes or chat-app profile without a memory switch gets memory: on; an explicit value stays.

To upgrade a box you installed with --setup, check out 0.9.1 and switch to your profile again. Its files are on disk already, so nothing downloads twice:

curl -fsSLO https://raw.githubusercontent.com/0xBakeer/TandemLLM/v0.9.1/install.sh
bash install.sh --setup --force-setup --yes
~/TandemLLM/run/ops/use-profile.sh <profile>

A box installed without --setup upgrades with bash install.sh --yes and the same --dir. On a new DGX Spark:

curl -fsSLO https://raw.githubusercontent.com/0xBakeer/TandemLLM/v0.9.1/install.sh
bash install.sh --setup --yes
~/TandemLLM/run/ops/use-profile.sh lfm25-exl3-3.5 --dry-run   # each file's URL and size
~/TandemLLM/run/ops/use-profile.sh lfm25-exl3-3.5

Known limits

• DuckDuckGo can answer with a rate limit or a bot check (HTTP 202). The tool returns that as an error result. SearXNG needs an instance with the json format on.
• The loop guard catches a repeated call. A model that repeats a failing approach with different calls is not caught yet.
• Carried over from 0.9.0: LFM2.5 has no draft head, and its shadow reader and KV pool are off. On LFM2.5 the conductor adds no memory notes. The Chat tab is in beta, there is no video input to the model, and the Flash-Next 5.05 pack does not load yet.

TandemLLM 0.9.0 2026-10-14 documentation

TandemLLM 0.9.0 adds a third model, Liquid AI's LFM2.5-24B-A2B, beside Qwen3.8-27B and Qwen3.8-Flash-Next. The Qwen models write the same text at the same speed as in 0.8.3. Every change is in CHANGELOG.md, and the PDF below is the manual.

What's new

• LFM2.5-24B-A2B runs on the engine: 24B parameters, 4 of 64 experts a token, a 131,072-token context and three slots. The decode step, the verify block and the commit each run as one CUDA graph.
• Seven LFM2.5 profiles: Liquid's BF16 checkpoint and six EXL3 packs from 2.5 to 6.0 bits a weight. The default is lfm25-exl3-3.5, the fastest in chat whose top token matches BF16's on at least 85 % of chat positions.
• The lookup drafts with no draft head. Edit and copy asks run 1.8 to 3.6 times faster, and the text equals plain decoding.
• The shared EXL3 kernels read half-integer rates (1.5, 2.5 and 3.5 bits a weight), as exllamav3 1.6.0 writes them below 4 bits. At whole rates they give the same bits as before.
• The conductor, Laya and the narrator are on for LFM2.5, fitted to the model. On 2.5, four short asks used 2,234 reasoning tokens, against 2,155 with no conductor and 9,098 with the Qwen router as written.
• Chat tab: the plus menu is no longer cut off, a page that a tool wrote opens from its tool card, and the Context panel counts every tool call, file, package, KV row and the resident prefix, or says why a figure is missing.
• Fixes: /v1/completions with a text prompt and two or more slots no longer fails with 500, and a pack stored as one model.safetensors counts as complete.

LFM2.5-24B-A2B profiles

One DGX Spark, 2026-10-14. Chat: one greedy request through the server, 512 tokens, thinking off, plain decoding. Top-1: how often the top token matches BF16's, from exllamav3's model_diff on oasst1 chat.

The six packs are one Hugging Face collection: TandemLLM LFM2.5-24B-A2B EXL3. They are public, and a download needs no token.

Install

On a new DGX Spark:

curl -fsSLO https://raw.githubusercontent.com/0xBakeer/TandemLLM/v0.9.0/install.sh
bash install.sh --setup --yes
~/TandemLLM/run/ops/use-profile.sh lfm25-exl3-3.5 --dry-run   # each file's URL and size
~/TandemLLM/run/ops/use-profile.sh lfm25-exl3-3.5

To upgrade a box you installed with --setup, check out 0.9.0 and switch to your profile again. Its files are on disk already, so nothing downloads twice:

curl -fsSLO https://raw.githubusercontent.com/0xBakeer/TandemLLM/v0.9.0/install.sh
bash install.sh --setup --force-setup --yes
~/TandemLLM/run/ops/use-profile.sh <profile>

A box installed without --setup upgrades with bash install.sh --yes and the same --dir. Saved Qwen conversations carry over.

Known limits

• LFM2.5 has no draft head yet. Liquid has not published its drafter, so the lookup drafts alone.
• The shadow reader is off on LFM2.5, because it reads Qwen's token ids.
• The KV pool is off on LFM2.5 and not verified there. Each slot reserves its window at load.
• On LFM2.5 the conductor adds only the class's core skill. General packages, effort sentences and memory notes stay out until they can go somewhere other than the end of the user's message.
• Laya stays below its 0.8 confidence bar on many short chats. Those turns run lean, at the profile's default level.
• At the same bit rate the 24B packs lose more to BF16 (KL) than the 8B sibling does. The cause is not known.
• Carried over from 0.8.3: the Chat tab is in beta, there is no video input, and the Flash-Next 5.05 pack does not load yet.

Credits

• Liquid AI for LFM2.5-24B-A2B. The packs are quantizations of it and carry its license, the LFM Open License v1.0.
• turboderp for exllamav3, which made the packs. Our EXL3 decode matches its LinearEXL3 bit for bit.

TandemLLM 0.8.3 2026-10-13 documentation

TandemLLM jumps from 0.3.0 to 0.8.3 in this release. These notes cover everything in between; the full history is in CHANGELOG.md, and the manual is attached as a PDF.

Models and weights

• Qwen3.8-Flash-Next runs on the same engine as the 27B, with the full 262,144-token window, image input and up to 3 chats at once.
• Weight profiles: the 27B runs in NVFP4 or as an EXL3 quant from 2.5 to 6.0 bits a weight. ops/use-profile.sh <name> switches between them.
• Six EXL3 quants of the 27B and their drafters are public on Hugging Face, in one collection.
• Community weights: anyone can add a quant with one JSON file in a pull request, and the setup page fetches the list from GitHub (docs/community-weights.md). The first one is ThinkingCap by @0xWhiteMage.
• A complete EXL3 pack of the 27B family loads directly as the model (#1 by @0xWhiteMage).
• A new DGX Spark downloads every file a profile needs with no Hugging Face token, and checks each one against the hub.

Speed and memory

• Drafting is priced per context length. Where a drafted round costs more than a plain step, the engine takes the plain step, so speculation is never slower than plain decoding.
• Flash-Next drafts with its own draft layer, and text lookup goes into the same round.
• One shared KV pool: every slot can reach 262,144 tokens, and memory is mapped as a conversation grows.
• The small helper models give memory back when it runs short and come back later.
• Decoding no longer stops while another chat prefills.
• Saved conversations survive restarts and upgrades: the disk cache identifies its files by content.

The dashboard

• A setup guide with Wren, a small model that explains each step and can fill in the form for you.
• A Chat tab (beta). A run keeps going when the browser closes, and another device picks it up mid-answer. Tools run in a sandbox, with terminal, preview and file panes, MCP servers, skills, plan mode with rewind, and Ask, Auto or YOLO modes. Long runs compact their context, and a panel shows what fills it.
• A phone layout: four tabs plus More, and a composer with every control in view.

The conductor

• Before a turn reaches the model, the conductor picks the task class, the thinking effort, the packages and skills and the memory notes, and guards the answer as it streams.
• Every client sees one model, TandemLLM, and it names the model it really runs.
• Other backends (DeepSeek, vLLM, SGLang) sit behind the same proxy and ledger.

Known limitations

• The Chat tab is in beta. The agent loop gets its next round of work in the coming release.
• No video input. The Flash-Next 5.05 pack doesn't load yet.
• Several chats at once aren't faster in total than one after another, only quicker to start.

Install

curl -fsSLO https://raw.githubusercontent.com/0xBakeer/TandemLLM/v0.8.3/install.sh
bash install.sh --setup --yes
~/TandemLLM/run/ops/use-profile.sh flashnext-3.05

Thanks to @0xWhiteMage for the first outside pull request and the first community quant.