4The dashboard and proxy on their own

The dashboard also runs without the engine, in front of any OpenAI-compatible engine or cloud API. Clients send their requests to one address; the proxy routes each one by its model id to the backend that serves it, copies the answer back byte for byte, and records tokens, time to first token, decode speed, cached tokens and cost in one ledger. It needs Python 3.11 or newer and one package, cryptography. No GPU, no torch.

4.1 Installing from a checkout

git clone https://github.com/0xBakeer/TandemLLM.git && cd TandemLLM
pip install '.[host]'      # the host extra adds psutil and NVML for the System view
tandem-dash                # http://127.0.0.1:8080/dashboard/ opens on Setup

The distribution is called tandemllm. Its host extra adds psutil and nvidia-ml-py, which the System view uses to read the machine; without them the collector falls back to /proc and nvidia-smi. The package is not on PyPI; install it from the checkout, or build the wheel with python -m build --wheel. python -m proxy in a checkout does the same as tandem-dash without installing anything but cryptography.

The page opens on Setup while no backend exists: add one, test the connection, save. The panels that only an engine fills (caches, speculation, the engine side of the conductor) say that this server has no engine.

4.2 The tandem-dash command

4.3 A first backend

A backend can be added in the GUI or declared in an env file. The file form suits a headless machine or a container. A local vLLM:

cat > dash.env <<'EOF'
TANDEM_BACKENDS=vllm1
TANDEM_BACKEND_VLLM1_KIND=vllm
TANDEM_BACKEND_VLLM1_URL=http://127.0.0.1:8100/v1
EOF
tandem-dash check --env-file dash.env      # prints what would run, writes nothing
tandem-dash serve --env-file dash.env

A cloud API, with the key in the environment, where it stays in memory and is never written to the database:

printf 'TANDEM_BACKENDS=deepseek\nTANDEM_BACKEND_DEEPSEEK_KIND=deepseek\nTANDEM_BACKEND_DEEPSEEK_KEY=    "$DEEPSEEK_API_KEY" > dash.env && chmod 600 dash.env
tandem-dash serve --env-file dash.env &
curl -s localhost:8080/v1/chat/completions -H 'Content-Type: application/json' \
    -d '{"model": "deepseek-flash", "messages": [{"role": "user", "content": "Say hi"}]}'

Then open http://127.0.0.1:8080/dashboard/#/usage: the request is one row there, with its tokens, its time to first token and its cost at the model’s price. Chapter 5 has a section for each kind of backend.

4.4 In docker

ops/Dockerfile.dash builds an image on python:3.12-slim from the same wheel, with the host extra and the built page inside, and no compiler or torch. It is configured by an env file; ops/dash.env.example is one to start from.

docker build -f ops/Dockerfile.dash -t tandemllm-dash .
cp ops/dash.env.example dash.env          # fill in; no quotes around values
docker run -d --name tandem-dash -p 127.0.0.1:8080:8080 -v tandem-dash:/data \
    --env-file dash.env tandemllm-dash

The config database, the usage ledger and the key file live in the /data volume. Inside the container the server binds 0.0.0.0, so what the outside reaches is what -p publishes: 127.0.0.1:8080 for this machine only. Keep it that way: no admin password is set by default, and this release has no sign-in screen (Section 3.8). The image runs as an unprivileged user and answers its health check on /health. The release’s tests build and install the wheel; they do not build the image.

Inside a container the host collector sees the container, not the machine. Run it with --pid=host (and --gpus all for NVML) for the machine’s view, or run tandem-dash hostcollect on the machine itself.

4.5 Where the proxy keeps its files

4.6 The proxy inside the engine’s server

The engine’s own server can relay too. Start it with --backends <config database> (QSE_BACKENDS in serve.env; empty means off). It then serves its own model as before and relays every request whose model belongs to a backend. A relayed request runs on its handler’s own thread and never takes the engine’s lock or a place in its queue, so a slow cloud answer cannot hold up the engine, and the engine’s queue cannot hold up a cloud answer. With --backends off the engine loads none of the proxy’s code and serves exactly as without it.

4.7 Access

The standalone proxy follows the engine’s access rule (Section 3.8) with its own variable, TANDEM_ADMIN_TOKEN, and adds user keys for the OpenAI routes (Section 5.12). It binds 127.0.0.1 by default.