Release notes, v0.11.0
TandemLLM 0.11.0 adds a fifth model, NVIDIA's Nemotron 3.5 Super VL 120B-A12B. It runs from our EXL3 packs at 4.0 and 2.0 bits a weight or from NVIDIA's own NVFP4 release, and every profile drafts with NVIDIA's LiLiCorr drafter for Super VL. The Qwen, LFM2.5 and Bolt models write the same text at the same speed as in 0.10.0, and their saved chats carry over. Every change is in CHANGELOG.md, and the PDF below is the manual.
Nemotron 3.5 Super VL 120B-A12B
Super VL has Bolt's hybrid layout at a larger size: 40 Mamba2 layers, 40 mixture-of-experts layers and 8 attention layers, with 22 of 512 experts per token. The engine now reads NVIDIA's ModelOpt checkpoints as stored, so NVIDIA's NVFP4 release runs without a conversion: NVFP4 experts, FP8 Mamba2 and attention projections and an FP8 head.
| profile | download | KL vs BF16 | top-1 vs BF16 | plain decode, tok/s |
|---|---|---|---|---|
nemotron-super-vl-exl3-4.0 (default) |
65.0 GB | 0.049 | 92.9 % | 31.1 |
nemotron-super-vl-exl3-2.0 |
36.84 GB | 0.325 | 82.1 % | 36.5 |
nemotron-super-vl-nvfp4 (NVIDIA's NVFP4) |
75.06 GB | 0.041 | 93.4 % | 19.6 |
One DGX Spark, 2026-10-16. Quality: 8 wikitext-2 rows of 1,024 tokens against the logits of NVIDIA's BF16 checkpoint. Decode: one request through the server, 512 greedy tokens, the mean of 3 runs.
Use the 4.0 pack. Its KL is 0.008 above NVIDIA's NVFP4 release, and it decodes 1.6 times as fast. With LiLiCorr and the lookup it writes 53.6 tok/s on code, and edit and copy asks reach 101.0 and 108.3 tok/s. The 2.0 pack is 17 % faster plain, but its KL is more than six times the 4.0 pack's. Take it only when 4.0 does not fit.
Both packs are public repositories on Hugging Face, 0xBakeer/TandemLLM-Nemotron-3.5-Super-VL-120B-A12B-EXL3-<bpw>, pinned by commit, and both are in one collection. The NVFP4 release and the drafter come from NVIDIA's repositories. docs/nemotron-h.md has the comparison with vLLM on the same NVFP4 weights.
Upgrade
Run the 0.11.0 installer again, then switch to Super VL. On a box you installed with --setup:
curl -fsSLO https://raw.githubusercontent.com/0xBakeer/TandemLLM/v0.11.0/install.sh
bash install.sh --setup --force-setup --yes
~/TandemLLM/run/ops/use-profile.sh nemotron-super-vl-exl3-4.0 --dry-run # each file's URL and size
~/TandemLLM/run/ops/use-profile.sh nemotron-super-vl-exl3-4.0
A box installed without --setup upgrades with bash install.sh --yes and the same --dir. A start of the 4.0 profile needs 90 GiB of free memory.
Known limits
- Super VL is text only. Its vision tower is not served yet, and neither is its MTP head.
- TandemLLM reads a prompt at 476 to 589 tok/s. vLLM reads the same prompts 4 to 5 times as fast.
- With a drafter, requests at once take turns. Three at once run at 19.6 tok/s in total on the 4.0 pack.
Files of the release
- tandemllm-manual-0.11.0.pdf (4466 KiB)