Community weights, TandemLLM v0.9.2 documentation

A community pack is a set of weights someone else published on Hugging Face that TandemLLM can load: a smaller quantisation, or a fine-tune of a model we serve. The setup guide lists each one under its model, after our own profiles, with the author's GitHub avatar, the format, the size and the speed a maintainer measured.

Listing a pack takes one JSON file. You send a pull request that adds models/<model>/community/<id>.json, and nothing else.

What happens to your file

You open a pull request with the file. The community check on the pull request runs python -m server.setup.registry --check and python -m server.setup.community --index --check.
A maintainer reads it, then downloads the pack at your commit and runs it on a DGX Spark. If it loads and answers, the maintainer fills verified and sets status to verified. If it does not, the entry stays submitted, or becomes broken with a note.
After the merge, every TandemLLM box sees the pack the next time someone opens the weights step of the setup guide. The box asks GitHub for models/community-index.json on the main branch. When GitHub does not answer, the box uses its last copy, then the copy that came with its release.

To have a maintainer write the file for you, open an issue with the "Add a community weight pack" form (.github/ISSUE_TEMPLATE/community-quant.yml). The setup guide's "Add yours" button opens that form with the model already filled in.

The file, field by field

The file name is the id: models/qwen3.8-27b/community/thinkingcap-27b-kearuga-exl3-4.00.json holds the entry with "id": "thinkingcap-27b-kearuga-exl3-4.00". That file is the first entry and a good one to copy. The JSON Schema is models/schema/community.schema.json.

Field	Required	What it holds
`schema`	yes	`1`
`id`	yes	The file name without `.json`. Lowercase letters and digits, with single dots or dashes between them, at most 80 characters.
`name`	yes	The name the card shows and the model calls itself when asked, at most 120 characters.
`base_model`	yes	The TandemLLM model it runs on, which is the folder it sits in. Today only `qwen3.8-27b`.
`finetune_of`	no	The Hugging Face id of the fine-tune, or `null` for a quantisation of the stock model.
`author.github`	yes	Your GitHub user name. The card shows `https://github.com/<you>.png` and links to your profile.
`author.name`	no	The name you go by.
`author.social`	no	Any of `x`, `huggingface` and `bluesky` as handles (`0xWhiteMage`, `name.bsky.social`), and `mastodon` and `website` as https URLs. No other keys.
`hf.repo`	yes	`owner/repo` on Hugging Face. It must be public: a box downloads it without a token.
`hf.revision`	yes	The full 40-character commit. A branch name is refused, because a branch can change after the review.
`format`	yes	`exl3`. The schema also knows `nvfp4`, `fp8` and `gguf`, and the check refuses them until TandemLLM can load them.
`bpw`	yes	Bits per weight, between 1 and 16.
`size_gb`	yes	The download in GB (10^9 bytes): the sum of the repository's file sizes at that commit.
`load`	yes	`standalone` when the pack is the whole model, or `overlay` when it holds EXL3 projections that go over the base checkpoint.
`modalities`	yes	`text`, `image`, `audio` and `video`, each `true` or `false`: what works in TandemLLM today.
`modalities_in_pack`	no	What the pack contains, even when TandemLLM cannot serve it yet. The card shows those in grey.
`drafters`	yes	`tandem-exl3` for our EXL3 draft models (K3 drafters and K4 draft head), `dflash2` for our BF16 draft models, or `none`.
`license`	yes	The license as its SPDX names, e.g. `Apache-2.0`.
`license_url`	no	An https link to the license file, best at your commit.
`description`	no	One or two plain sentences, at most 500 characters. The card shows it as text: no HTML, no Markdown.
`tags`	no	Up to 12 lowercase words, such as `fine-tune` or `reasoning`.
`added`	yes	The date of the pull request, `YYYY-MM-DD`.
`status`	yes	`submitted`. A maintainer changes it.
`verified`	no	Filled by the maintainer who ran it: `build`, `device`, `date`, tok/s per kind of text (`prose`, `code`, `chat`, `plain`), tokens per round, and a note.
`links`	no	https links, such as `pr` for a pull request that made the pack load.

The check refuses a modality the load path cannot serve. A standalone pack is text only for now (ENG-271), even when it carries a vision tower: write "image": false and list image in modalities_in_pack. An overlay keeps the base checkpoint's vision tower, so "image": true is allowed there. No path serves audio or video yet.

What is not accepted

A pack that is private or gated, or that needs a token to download.
A branch or tag in hf.revision instead of a commit.
A format other than EXL3, until a loader for it is in TandemLLM.
A base model other than qwen3.8-27b.
A claim in modalities that the load path does not serve.
Links that are not https, and HTML or Markdown in any text field.
A license that does not let others download and run the weights.
A pull request that changes anything besides the one JSON file. A maintainer regenerates the index models/community-index.json, or you run python -m server.setup.community --index.

Running a pack

On the box, ops/use-profile.sh community/<id> writes the pack's profile from its entry, downloads the pack at the pinned commit into ~/.tandemllm/community/<id>, checks every weight file against the sha256 Hugging Face publishes, and fetches our draft models if they are missing. Then it stops the engine, starts it on the pack and sends one test chat. It refuses a download that would leave less than 60 GB of disk free, and it rolls back to the previous profile when the start or the test chat fails. --dry-run prints every URL and size without downloading. The setup guide's "Use" button runs the same steps after its review page.

The engine estimates a pack's memory from our own profile of the same format and the nearest size (exl3-4.0 for a 4.0 bpw EXL3 pack). The number is only an estimate until a maintainer has loaded the pack.