LFM2.5-24B-A2B

LiquidAI's LFM2.5-24B-A2B is the third model TandemLLM runs, beside Qwen3.8-27B and Qwen3.8-Flash-Next. Support ships in release 0.9.0. The model has 24B parameters; 4 of its 64 experts run for each token. It is a chat model and the fastest model on this engine.

Code: engine/models/lfm2_moe/ (registry name lfm2_moe, selected by Lfm2MoeForCausalLM in config.json). Setup Guide entry: models/lfm2.5-24b-a2b/model.yaml. Measurements: models/lfm2.5-24b-a2b/measurements/0.8.3-summary.json.

The model

A decode token reads 4.9 GB of the BF16 weights: experts 2.87 GB, convolution projections 1.01, head 0.52, dense MLPs 0.29, attention 0.21 (PackIndex.decode_bytes). The decode step runs about 400 kernels, so it is one CUDA graph.

Profiles

DGX Spark, 2026-10-14. Decode step: tools/lfm_check.py bench, one request, 256 tokens, from its graph. Chat: one greedy request through the server, 512 tokens, thinking off, plain / lookup. Quality: exllamav3 model_diff against BF16, 20 rows of 2,048 tokens of oasst1 chat and of wikitext-2. Engine KL: tools/lfm_quality.py, 100 rows of oasst1 chat.

• Packs: exllamav3 1.6.0, mul1 codebook, head 6 bits (8 for 6.0), default calibration. One Hugging Face repository a size, 0xBakeer/TandemLLM-LFM2.5-24B-A2B-EXL3-<bpw>, pinned by commit (QSE_WEIGHTS_REVISION). The repositories are public; a download needs no token. Every file is checked against the hub's sha256.
• Default: lfm25-exl3-3.5. Rule: the fastest profile in chat as served (lookup on) with chat top-1 against BF16 of at least 85 %. 3.5: 143.7 tok/s at 86.7 %. 3.0: 168.5 tok/s at 83.6 %, below the floor.
• Setup Guide tile: the fastest cell of the default profile's served row, 147.5 tok/s (3.5, prose).
• -hq 4.0 (attention and convolution projections +2 bits, dense MLPs +1, 4.06 bpw, 12.81 GB): chat KL 0.082, top-1 89.7 %; served chat 131.2 tok/s against 144.0 for 4.0 and 127.6 for 4.5 in one window. Not offered.
• The KL values are high for these rates. The same recipe on the 8B sibling at 4.10 bpw gives KL 0.0955 against 0.0862 for a reference conversion. The cause on the 24B is not known.
• Half-integer rates: below 4 bpw exllamav3 1.6.0 gives some tensors 1.5, 2.5 or 3.5 bits a weight (a tile is 16 K + 8 int16 wide; an odd weight takes one bit more than an even one). The 2.5 and 3.5 packs use them. The EXL3 kernels decode them as exllamav3 does (tools/exl3_linear.py, dq8_frac; bit codes 15, 25, 35) in the decode step, a verify block and the prefill. The pack reader refuses an unsupported rate or codebook by tensor name.

Switch: ops/use-profile.sh lfm25-exl3-3.5, or the Setup Guide (#/setup).

Chat, reasoning and tools

• Template: the checkpoint's chat_template.jinja. It opens <think> at the end of every prompt; the server splits reasoning from the answer as for Qwen (server.md).
• Thinking off (chat_template_kwargs: {"enable_thinking": false}, or the conductor's): the server closes the block right after the generation prompt.
• Tool calls: a Python list between <|tool_call_start|> and <|tool_call_end|>, for example <|tool_call_start|>[get_weather(city='Berlin', days=3)]<|tool_call_end|>. server/toolcall.py (PythonicCalls) rewrites each block into the JSON tool tier as the text arrives; the client gets OpenAI tool_calls. Values are Python literals (and true, false, null), read with ast.literal_eval, never run. A block that is not a list of keyword calls stays text.
• Sampling default when a request names none: temperature 0.1, top_k 50, repetition penalty 1.05 (TEMPERATURE, TOP_K, REP_PENALTY).
• Conductor: on by default, with the model fit below (Conductor).

Conductor

The conductor (Laya's three decision models, the narrator, the session monitor) is on by default. The router's documents target Qwen3.8; LFM2.5's adapter declares how the router fits it (Capabilities.conductor in engine/models/lfm2_moe/serving.py, read by server/conductor/modelfit.py; server.md):

Why, measured on lfm25-exl3-2.5, 2026-10-14 (models/lfm2.5-24b-a2b/measurements/conductor-0.9.0.json):

• With thinking off the model answers the router's class prompt as a chat request. Its one-id answers were right on 0 to 2 of 8 labelled requests at confidence 0.08 to 0.44 or wrong at up to 0.99. Laya stays under its 0.8 bar on short chats, so a short chat fell back to the profile's default class: chat through curl (everyday packages and memory notes, 2,036 to 2,347 hidden tokens), code_edit at xhigh in the Chat tab (coding packages, 5,825 to 6,022).
• The router puts its notes at the tail of the user's message. The same note texts on the raw path, 4 short asks, greedy: 5,817 reasoning tokens at the tail, 2,969 as the system message, 2,155 without them. The package note alone at the tail: 7,363; the memory notes alone: 2,614. The reasoning follows the package's checklist line by line; with thinking off it moves into the answer.
• Chat-tab tasks (5 tasks, 2 runs each, reasoning tokens): the fit with the class rules as its only note (75 to 84 tokens) 3,083 and 6,416 in two windows; with the effort sentences added 8,343; with the general packages added 12,934 (5,788 to 7,654 hidden tokens a turn); the router as written 13,426. The raw path in the same windows: 2,748 to 3,670. Runs at temperature 0.1 vary.

The model fit against the raw path and the router as written (QWEN38_CONDUCTOR_FIT=off). Same server, one request at a time; short asks greedy at 8,192 tokens at most; tasks at the sampling defaults (temperature 0.1), each run twice:

The follow-up rows ran on the :8000 server after the deploy (raw and fit) and on the side server (router as written); there the router's testing packages had the model check the page through the shell: 15 shell calls in the follow-ups and 13 tool errors over the 6 runs. The raw column is the model fit's window; in the router's window the raw tasks gave 14/14, 7,352, 0. Rates in the model fit's window (no other request on the box): 171.6 tok/s on the short asks against 166.7 raw, 161.1 on the tasks against 165.7. The tasks: files, a Python script, a search, an HTML canvas animation, an analog clock, a landing page, a JSON edit; the check reads the files each run wrote.

Not yet: the packages, the memory notes and the effort sentences at a place this model does not read as part of the request (the system message cost 1.38 times the raw thinking where the tail cost 2.7 times); Laya's class question on short chats (below its bar, so they fall back).

How the engine runs it

• One row path: the decode step and every row of a verify block run the same kernels a row at a time (engine/models/lfm2_moe/kernels.py). EXL3 projections use tools/fn_kernels.dense, experts its MoE bank kernels, BF16 projections a 16-row GEMM. A verify row has the decode step's bits, so greedy speculation writes the plain greedy text.
• Self-check at load: a recorded 24-token decode verified as one block, every row compared bit for bit. A difference turns speculation off.
• Rollback: a verify block writes nothing into the convolution state. The commit rebuilds the state from the accepted path's last two inputs and moves the path's KV rows. Rollback is exact.
• Prefill: chunks of 4,096 tokens on batched kernels (cuBLAS or the EXL3 GEMM, experts grouped, SDPA with a lower-right causal mask). The next token agrees with the row path; the bits do not. 8,192 tokens prefill at 7,360 tok/s on 4.0.
• Dense splits: ops/lfm25-dense-splits.json, from tools/fn_tiles.py --dense at each pack's shapes. Decode step with / without, tok/s: 2.5 188.2 / 171.2, 3.0 175.0 / 159.1, 3.5 153.5 / 143.4, 4.0 145.9 / 136.1, 4.5 128.8 / 121.6, 6.0 106.9 / 106.9. The graph saves 5 % against eager launches.

Speed by kind of text

One greedy request a cell through the server, 512 tokens, thinking off, suffix store off, plain / lookup, tok/s. Edit: a file back with one name renamed. Copy: a text back unchanged. Every lookup text equals the plain one.

The lookup costs up to 5 % in prose and chat and takes edit and copy asks to 1.8 to 3.6 times the plain rate.

Verify staircase, 4.0, chain blocks from their graphs, median of 20, 1,024 tokens of context, before the dense splits (tools/lfm_check.py staircase):

Each row adds about 3 ms because a block reads the union of its rows' experts. At 8,192 tokens of context the blocks cost 0.5 to 3.5 ms more. A block pays when it is accepted for about 1.4 tokens a row.

Speculation

• Drafter: the lookup alone (the request's text and the suffix store) through the shared tree drafter with no head (engine/methods/mtp_drafter.py NoHead), cut by StairCut.
• Prices: one file a profile, ops/prices/lfm2.5-24b-a2b/<profile>.dgx-spark.json, from its staircase and bench (tools/lfm_check.py prices).
• No-head rules: a round with no lookup match is the plain step and is priced as one; each round's own match decides through the cut's calibrated acceptance; the class-wide skip is not consulted. The lookup's confidence carries its match length, so a long copy match scores above a short phrase match.
• Identity: tools/fn_server_check.py --compare on 4.0, lookup against plain, 20 requests (greedy, thinking, streaming, every tool_choice, JSON, penalties with logit_bias, logprobs, seeds, a stop string): every answer equal, largest logprob difference 0.0.
• A draft head plugs in through the adapter's draft_heads() in place of NoHead.

Parallel requests and caches

• Three sequence slots over one copy of the weights (QSE_PARALLEL=3). A slot taking a plain step joins the round as one row (joint_plain); when every decoding slot does, the round is one joint decode step from a graph (JointDecodeGraph).
• 4.0, lookup on, store off, 256 tokens each (tools/lfm_parallel.py): 1, 2 and 3 requests at once decode 139.9, 169.1 and 177.8 tok/s in total; every text equals its request alone.
• Resident prefix and state store: 30 convolution states (240 KiB) as sections, K and V of the 10 attention layers as views, 4,096-token grid. A warm answer equals the cold one with equal logprobs at 4,096 and 12,288 tokens (tools/fn_cache_check.py).
• Disk tier: 12,288 tokens restored in 0.96 s against 2.6 s cold, answer and logprobs equal. Session spill: 13,650 tokens drained in 309 ms (0.19 GB), read back in 163 ms. One tier folder a pack.

Not supported yet

• A draft head (none published for this model).
• Vision: the model reads text only.
• The shadow reader: it reads Qwen's token ids and was validated on Qwen text; off on this model.
• The KV pool: off in the profiles (QSE_KV_POOL=off), not verified on this model. Each slot reserves its window at load.

Checks

A near tie: the reference's two best tokens are within 8 bf16 steps of its largest logit (exactness.md).