33HTTP API reference

The engine’s server and the standalone proxy speak the same API for clients, and each adds its own routes for the dashboard and the conductor. This chapter lists every route, every header and every error they use. The tables are written in manual/tools/http_ref.py, and gen_reference.py --check compares them with the code on every build: a route a handler tests, a header name or an error type the code uses that is missing here, or a row the code no longer has, fails the check.

33.1 Who may call what

The admin password is the admin token, QSE_ADMIN_TOKEN in secrets.env (TANDEM_ADMIN_TOKEN for the standalone proxy). It is not set by default. Then every route below answers anyone who reaches the port, and only /metrics keeps the metrics token when one is set (Section 3.8). With a password set, an admin route needs it as a bearer header or a dashboard session (a cookie the token buys). A request from the machine itself (loopback, no proxy headers) needs neither, because the watchdog and the measurement tools rely on that (--trust-loopback).

The OpenAI routes are open on the engine, with or without a password. Behind the proxy they are open until the first user key exists; from then on every request from another machine needs a key. Table 33.1 names the labels the route tables use.

33.2 The engine’s routes

Table 33.2 lists the engine server’s routes. A route the server does not know answers 404. A route whose part is off on this server also answers 404: the conductor’s routes without --conductor-config and the proxy’s without --backends, each with a message that names the flag, and the decision and Laya routes without decision models, as an unknown route. With --backends the engine’s server also serves the proxy’s routes.

33.3 The proxy’s own routes

Table 33.3 lists the routes of tandem-dash, the standalone proxy. It serves no model of its own: every completion goes to a backend, and the answer’s bytes go back to the client unchanged. Table 33.4 adds the two small servers that run beside the engine or the proxy: the remote host collector and the status responder that answers while the engine is stopped.

33.4 The conductor’s and the configuration routes

The conductor’s routes and the proxy’s configuration routes are generated from the route tables at the top of server/conductor/routes.py and proxy/config_api.py.

33.5 Headers

Table 33.7 lists every header the engine and the proxy read or write beyond the ones HTTP itself needs. in is a header the server reads, out one it writes.

33.6 Errors

Errors are OpenAI-shaped:

{"error": {"message": "max_tokens must be at least 1", "type": "invalid_request_error",
           "param": "max_tokens", "code": null}}

A 400 names the field or the message part it refuses in param. A refusal the client can retry (a full queue, the memory gate, a server that drains) has the type server_busy, the status in code, and a Retry-After header. A refusal after a stream opened (a queue timeout, a prompt too long, the memory gate, a backend that failed) ends the stream with a chunk whose finish_reason is "error" and which carries the error object, then [DONE]. The decision routes keep their own body, {"type": "error", "error": {"type", "message"}}. Table 33.8 lists every status and type.