Contents TandemLLM Docs
v0.11.1
Star58

Documentation

TandemLLM 0.11.1

released 2026-10-16. The manual, chapter by chapter, with the reference tables, the release notes and the pages of docs/.

What is new in 0.11.1 The manual as one PDF Source on GitHub

IIntroduction

Two chapters for every reader. Chapter 1 says what TandemLLM is, names the machine and the model every number in this book comes from, says what the repository contains and what runs today, and states the two rules every change has to pass. Chapter 2 follows one chat request through all of it, from the proxy to the stream going back, and names the chapter that explains each step.

  1. 1What TandemLLM is
  2. 2One request, end to end

IIInstalling and running

How to install and run it. Chapter 3 starts with the setups that work today, then installs the engine with one command, says who can reach it, and adds the small models and other backends. Chapter 4 installs the dashboard and proxy on any machine, Chapter 5 puts other engines and cloud APIs behind it, and Chapter 6 is the operator’s chapter: the memory rules, the service scripts, the watchdog, how to hold the board for a measurement, where the logs are, and how to upgrade and roll back. Chapter 7 runs weights other people published, and says how to list your own.

  1. 3Installing TandemLLM
  2. 4The dashboard and proxy on their own
  3. 5Other engines behind the proxy
  4. 6Operations
  5. 7Community weights

IIIHow the engine works

Why the engine is fast, and why its output is still the model’s own text. The chapters go from the model’s layers and the byte budget (8) through the weights (9), the kernels (10), the drafters and StairCut (11), the exactness contract (12), the caches (13), the server (14) and the measurement method (15) to the seams a new model plugs into (16), and ends with the second and third models served through them, Qwen3.8-Flash-Next (17) and LiquidAI’s LFM2.5-24B-A2B (18). Every number is labelled measured or arithmetic.

  1. 8Architecture
  2. 9Quantisation
  3. 10Kernels
  4. 11Drafters, StairCut and tree verify
  5. 12Exactness
  6. 13Caches
  7. 14Serving
  8. 15Measurement
  9. 16Adding a model
  10. 17Qwen3.8-Flash-Next
  11. 18LFM2.5-24B-A2B
  12. 19Nemotron 3.5 Bolt and Super VL

IVThe conductor

The conductor: the layer in front of the model that plans each turn and watches it run. Profiles and the router (20), packages, the skill library and the small decision models (21), the session monitor (22), status lines, the voice and the shadow reader (23), memory notes (24), and the same conductor in front of other backends (25). Nothing here changes the model’s arithmetic; all of it changes what the model reads.

  1. 20Profiles and the router
  2. 21Packages, the library and the deciders
  3. 22The session monitor
  4. 23Status lines, the voice and the shadow reader
  5. 24Memory notes
  6. 25The conductor in front of any backend

VThe dashboard

The dashboard, view by view (26), its Chat tab with the sandbox and the permission modes (27), and how it watches several engines at once: the Parallel page, the engines’ own metrics pages, the plugins inside vLLM and SGLang, and the host collector (28). Every figure on it is measured or null, never estimated.

  1. 26The dashboard, view by view
  2. 27The Chat tab
  3. 28Watching several engines

VIReference

The reference. Chapters 29 to 32 are generated from the code: every flag, variable, route, metric and ledger column, with the value the served profile sets. Then troubleshooting by symptom (33), questions and answers (34), the roadmap (35) and the changelog (36). The glossary and the index follow.

  1. 29Command-line reference
  2. 30Environment and configuration reference
  3. 31HTTP API reference
  4. 32Metrics and the ledger
  5. 33Troubleshooting
  6. 34Questions and answers
  7. 35Roadmap
  8. 36Changelog summary

About this manual

This book describes TandemLLM version 0.11.1: an environment that runs the models your agents need, on your own machine and in front of cloud APIs. It covers the engine with the models it runs together and the conductor that plans each turn. It covers the caches and the memory notes, and the dashboard and proxy that watch every backend. It sits beside the Markdown pages in docs/ and goes further: figures and screenshots show how the parts fit together, and a script writes its reference part from the code.

Part I says what TandemLLM is, names the machine and the model every measurement comes from, and follows one request through it. Part II installs and runs it, alone or together with other engines and cloud APIs. Part III explains how the engine reaches 49.89 tokens a second for one request on its test board, and how the benchmark row measures it. Part IV covers the conductor, the layer that plans each turn and watches it run. Part V walks through the dashboard’s views. Part VI is the reference: about 800 rows that a script writes from the code or checks against it, then troubleshooting and the roadmap. A glossary and an index close the book.

How numbers are labelled

Every number in this book is one of three kinds, and the text says which.

  • measured: measured on the test machine, a DGX Spark with Qwen3.8-27B (Section 1.1), with the published NVFP4 weights unless the text names another set. The caption or the sentence names the version or the date, and the tool that measured it when one did.
  • arithmetic: computed from sizes, such as bytes read divided by memory bandwidth. Nothing ran.
  • estimate: a guess with its basis stated. The book uses few of them and marks each one.

The speed numbers of the StairCut paper are the reference for single-request speed. They were measured on the published weights with the persistent suffix store off, on the benchmark row described in Chapter 15. Later releases are held to those numbers by a same-hour comparison before they ship.

Agreement with greedy decoding is a measured property with one documented exception, a near tie between the two best tokens. Chapter 12 states it precisely. Where this book says that resuming from a cache is “bit for bit”, a test compared the bytes.

Conventions

Commands, flags, variables, files and HTTP routes are set in monospace. Paths under ~/.qwen38-spark-engine/ and ~/.tandemllm/ are the engine’s and the proxy’s state directories on the machine that runs them. Runtime names still carry the project’s old prefix (QWEN38_, QSE_); a later release renames them together.

The engine runs on Linux. The dashboard and the proxy run on Linux and macOS.

Building this book

The source is in manual/ of the repository. make in that directory regenerates the reference tables from the code (tools/gen_reference.py) and runs latexmk -lualatex. The repository keeps one built copy, manual/tandemllm-manual.pdf, so a reader needs no LaTeX; build it again before a release changes the book. The manual.yml workflow also builds it when a release tag is pushed and attaches it to that GitHub release, as tandemllm-manual-<version>.pdf.

Citing

Cite the paper for the decoding method and its numbers (Section 1.7), and the repository for the software; CITATION.cff has both.

License

The text of this manual is licensed under CC BY 4.0, like the documentation in docs/. The code it describes is licensed separately: AGPL-3.0-only, or a commercial license from the copyright holder. The model weights are not part of the repository; the base model and the base drafter are Apache-2.0, and so are the published NVFP4 overlays and fine-tuned drafters derived from them.

TandemLLM 0.11.1. © 2026 the TandemLLM authors.