Contents TandemLLM Docs
v0.13.0
Star58

Release notes, v0.13.0

TandemLLM 0.13.0 on GitHub, published 2026-10-21. The changelog chapter of this version: Changelog summary; the full file: CHANGELOG.md.

TandemLLM 0.13.0 adds Ornith AI's Ornith-1.5-122B, an agentic and coding model on the Qwen3.5 mixture-of-experts layout, from our EXL3 pack at 4.0 bits a weight. It reads images, thinks before it answers and drafts with its own MTP layer. Every change is in CHANGELOG.md, and the PDF below is the manual.

Ornith-1.5-122B

  • 0xBakeer/TandemLLM-Ornith-1.5-122B-EXL3-4.0 at 92f7483c, 64.6 GB, public. Every projection is at 4 bits, the head at 6, the MTP layer at 4 and the vision tower at 6.
  • exllamav3 1.6.2 converted it from the BF16 checkpoint. Its 13 experts with all-zero fp16 weights needed a small patch to the quantizer, which is in the repository.
  • ops/use-profile.sh ornith-1.5-122b-exl3-4.0 downloads it and starts it, and the Setup Guide lists it. A download needs no token. The base model, the pack and incoai's DFlash2 drafter for Ornith are in one collection.
  • The engine runs it on a new trunk, engine/models/qwen3_5_moe: 36 Gated DeltaNet and 12 gated attention layers, 8 of 256 experts and a shared one a token, on the same EXL3 kernels as Flash-Next. It serves one request at a time, with a window up to 262,144 tokens.
  • The profile drafts with the MTP layer and the lookup. After a round where StairCut drafted nothing, the engine takes 1, 2, 4, then 8 plain steps before it tries again. QSE_ORN_HEAD=dflash2 switches to the DFlash2 drafter.

Measured on one DGX Spark on 2026-10-20, a fresh server for each row, greedy, thinking off, 512 tokens, one run each. The last column is a 2,048-token answer that writes a whole HTML game:

tok/s prose code chat edit copy 2,048-token game
plain 37.6 38.4 37.8 38.0 38.1 37.2
MTP layer + lookup (default) 85.3 114.2 110.0 115.9 124.8 103.3
DFlash2 + lookup 67.3 91.7 92.2 106.9 122.8 96.8

Ornith-1.5-122B decode speed on one DGX Spark: plain, DFlash2 + lookup, MTP layer + lookup

Every drafted text is the plain run's. A start loads the pack in 65 s, and the weights take 61.16 GiB with the MTP layer.

Against the BF16 checkpoint's logits, on 8 wikitext rows of 1,024 tokens, the engine gets KL 0.139 and top-1 86.2 %. exllamav3's own forward of the pack gets KL 0.136 on the same rows. Against that forward on the same pack, the engine's top token agrees at 498 of 512 positions, so the two are close but not equal. Of the 14 differences, 9 have a margin under 0.5 logits and one a margin over 4. The cause is not tested.

measurements/0.13.0-summary.json has the runs. Requests at once, prompt reading speed, images and repeat runs were not measured.

On tool-eval-bench, one run a row on this release, Ornith scored:

row score time
raw, thinking off 93 9.8 min
raw, low / medium / high / xhigh 93 / 94 / 94 / 93 18.4 / 13.1 / 13.5 / 11.8 min
conductor, tools 96 12.2 min
conductor, chat 95 30.1 min
hard mode, raw / conductor 80 / 84 10.7 / 10.1 min

Ornith-1.5-122B tool-eval-bench scores and time a row on one DGX Spark

docs/tool-eval-bench.md has every row, the misses and the flags.

Other changes

  • The response cache keys on the effort level a request resolved to (SRV-645). A raw xhigh tool-eval-bench run had replayed 190 answers of the high run before it.
  • tools/teb_batch.sh and tools/teb_summary.py run tool-eval-bench rows one at a time and keep what each row ran under (OPS-209).
  • A package whose result an offered tool delivers can stay out of the turn (SRV-615). It ships in shadow: it logs what it would keep out and changes no prompt.
  • In the Chat tab, grep given one file's path searches that file (SRV-649), and read_file reaches every line of a file over 4 MiB (SRV-651).
  • tests/test_grammar.py passes every run (SRV-647).
  • Ornith's caches have room (SRV-655). At the first ceiling of 74 GiB every end anchor and store snapshot was refused; the limits are now 86 and 90 GiB with a 2 GiB pin, and a store decline names the memory policy and its figures.
  • When the end alignment fails on a rewritten tool call, the engine serves the part it aligned and resumes there (SRV-657). On Ornith one such turn had read 132,635 of 144,923 tokens again.
  • In the Chat tab, writes and long commands stay whole in later requests until a fold point (SRV-659), and plan mode and a tool check that times out keep the request's head (SRV-663). A run stopped at its 100-call cap leaves the chat idle, and render_page keeps a shader error's compiler lines (SRV-665).
  • Every test file runs its tests as a script, and a file that runs none fails (SRV-661). tools/chat_trace.py replays a recorded chat through chatd in seconds, and tools/prefix_check.py reads what each request reads again (SRV-667).

The other models write the same text as in 0.12.2 and keep their saved conversations. The router kernel now takes up to 512 experts, which moved the fingerprints of Flash-Next, LFM2.5 and NemotronH; at 512 it runs as before, and their first start adopts 0.12.2's files.

Update

Run the installer again: bash ~/TandemLLM/src/install.sh --yes --ref v0.13.0, then tandem restart and tandem chat restart. On a box you installed with --setup, add --setup --force-setup and run ops/use-profile.sh with your profile after it. For Ornith, run ops/use-profile.sh ornith-1.5-122b-exl3-4.0; it downloads 64.6 GB and needs 93 GiB of free memory to start.

Files of the release