Security proxy for agent and LLM traffic: isolates keys, filters what leaves, keeps access reliable
Each provider is an [upstream.<name>] table in reiki.toml. A client reaches it through Reiki at http://127.0.0.1:4097/<name>.
base_url to the root the provider’s own SDK uses by default.http://127.0.0.1:4097/<name>, so the SDK appends its usual paths and Reiki forwards them to base_url.auth_header and auth_format to the provider’s expectation (defaults authorization and Bearer {key}); a bad pair is refused when the config loads.Authorization: Bearer, x-api-key, x-goog-api-key or a ?key= query parameter, whichever their SDK uses. Headers win; the query parameter is last.A stream can chain keys from several providers; a request uses only keys from the provider in its path.
Two optional upstream settings adjust the JSON body before Reiki sends it:
model_aliases = { "gpt-4o" = "opencode/gpt-4o" } replaces the top-level model string when it matches a key.drop_params = ["stream_options"] removes the listed top-level keys.[upstream.zen]
base_url = "https://opencode.ai/zen/v1"
model_aliases = { "gpt-4o" = "opencode/gpt-4o" }
drop_params = ["stream_options"]
Reiki rewrites the body once before picking a key, so failover replays the same bytes. A non-object body, a missing or non-string model, an unknown model and a non-matching drop_params pass through byte-for-byte. Only top-level keys are dropped; nested objects stay. Keys and the model value are compared after JSON escapes are decoded, so {"model":"gpt\u002d4o"} matches an alias for gpt-4o; otherwise they must match exactly.
By default Reiki uses HTTP/1.1, asks for uncompressed responses and sends lowercase header names. Optional settings make the relay more of a plain pass-through:
http2 = true negotiates HTTP/2 when the upstream offers it over TLS; without TLS the connection stays HTTP/1.1.forward_accept_encoding = true sends the client’s Accept-Encoding upstream instead of identity; a client that sent none still gets identity. Reiki decodes gzip, deflate, brotli and zstd so error bodies can be read for failover and logging; the client gets plain bytes without Content-Encoding. A forward upstream relays content-encoding as-is.max_decoded_bytes caps a body after decoding, compressed responses included. Default 64 MiB, at least 1. A decoded non-SSE response over the cap becomes a 502 before anything reaches the client; an SSE response that passes the cap ends with an error. The compressed bytes read are capped too, at max(2x this cap, 1 MiB), so an endless body that decodes to nothing cannot be consumed without bound; exceeding it fails the response the same way.response_idle_timeout_secs drops a response body with no chunk for that many seconds. Default 300, at least 1. A buffered decoded body answers 504; a streamed body ends with an error.response_timeout_secs bounds the whole body from its head. Unset means 300 seconds for a body that is not an event stream, and no total limit for an event stream; at least 1 when set. An explicit value applies to event streams too. The separate 120-second request timeout bounds the wait for the head.title_case_headers = true sends Title-Case header names (Content-Type, X-Request-Id) on HTTP/1.1; HTTP/2 always lowercases, so it does nothing with http2 = true.send_keep_alive = true adds Connection: keep-alive. HTTP/1.1 connections persist without it, and Reiki never forwards the client’s Connection header, which only describes the client’s connection. A forward upstream always closes, so this is refused there.[upstream.zen]
base_url = "https://opencode.ai/zen/v1"
http2 = true
forward_accept_encoding = true
The credential header goes first, ahead of the client’s headers. These settings apply to reverse-proxy routes; an intercepted tunnel relays traffic as the client sent it (see tunnel).
They change how Reiki connects, not which requests an upstream accepts, and Reiki never adds, rewrites or invents client-identifying headers.
An upstream with a limits table gives each key an outbound budget; without one keys are never shaped:
[upstream.zen.limits]
preset = "zen-free" # or "openrouter-free"
rpm = 20 # requests per minute
tpm = 100000 # tokens per minute (initial estimate: body bytes / 4)
concurrent = 2 # requests in flight
daily_requests = 50
daily_tokens = 250000
max_wait_secs = 20 # longest Reiki waits for a slot, 1-120
Every field is optional; an explicit 0 means unlimited, even over a preset. A preset seeds the values and explicit fields override them; min_rpm sets a floor for learning.
Reiki takes a slot for the chosen key before sending. A wait within max_wait_secs is slept off, at most twice per key; a longer wait or a closed daily quota skips that key for this request and tries the next. If no key is usable, the client gets 429 with JSON error code reiki_shaped, Retry-After and x-reiki-origin: local. A concurrent slot is held until the response body ends or the client disconnects.
The first token charge is an estimate of body bytes / 4. When the response reports provider usage in JSON or an SSE event, Reiki replaces it with the prompt + completion count as the body streams, so tpm and the daily and monthly token counters include output. A body with no usage, or a compressed body, keeps the estimate.
A response from that key’s own upstream teaches the bucket: Retry-After on a rate-limit 429 pauses it, 402/quota exhaustion closes it for the key cooldown, and remaining: 0 closes it until the reported reset. A rate-limit or quota failure on a request that names a model only cools that route and model, so it does not pause or close the bucket; see shaping. The learned cap, remaining pause or closed time and current daily and monthly counters persist in the state file across restarts; the sliding minute log is not saved, and a key whose file or upstream changed starts with a fresh bucket.
A forward upstream hands each request to a listener or a program instead of sending it itself; that process can pass it on to another process, a queue, another harness or a proxy.
Socket target, an already-running listener:
[upstream.example.forward]
socket = "/run/forwarder.sock"
expect_uid = 1000
Command target, a program started per request:
[upstream.example.forward]
command = "/usr/local/bin/forward-request"
args = ["--config", "/etc/forward.toml"]
pass_env = ["FORWARD_TARGET"]
Set exactly one of socket or command; add streaming = true to relay the answer as it arrives (see below).
Reiki is the HTTP client and speaks plain HTTP/1.1. The request line uses the upstream URL’s path and query, and Host is the upstream host:
POST /v1/chat?x=1 HTTP/1.1
Host: example.com
X-Forwarded-Proto: https
Connection: close
Content-Length: 2
Authorization: Bearer <key>
Content-Type: application/json
{}
The other client headers pass through as for a normal upstream. The key is sent only in the upstream’s credential header, with no second copy and no Reiki-specific header; a passthrough upstream forwards the client’s own credential unchanged.
Client-sent Forwarded, X-Forwarded-For, X-Forwarded-Host, X-Forwarded-Proto and X-Real-IP are removed (Reiki sets X-Forwarded-Proto), as are hop-by-hop headers and headers named by the client’s Connection.
The status must be 200-599. Without streaming, the body is buffered whole: up to 8 MiB, 128 headers and 64 KiB per header value; chunked responses are joined. Reiki sets content-length to the buffered length (HEAD keeps the forwarder’s length; 204 and 304 carry none) and relays content-encoding without decoding it. Expect is not passed on and hop-by-hop headers are removed. Nothing reaches the client until the forwarder finishes.
streaming = true on [upstream.X.forward] (socket or command) hands the client the answer as it is produced, which event streams need:
[upstream.zen.forward]
command = "/usr/local/bin/bun"
args = ["/usr/local/lib/reiki/forwarder.ts", "--allow", "opencode.ai"]
streaming = true
content-length is kept; otherwise the body is chunked. transfer-encoding and the other hop-by-hop headers are removed, and content-encoding is still relayed as-is.command keeps its concurrency slot and process until the body ends or the client disconnects; a disconnecting client ends the child after the usual 1-second grace.Any failure of the link answers 502: a failed connect or start, a wrong uid, a connection closed without a response, a malformed or oversized response, a bad status or a timeout. Reiki logs it at error level without key material, does not cool the key, does not retry on the same or another key, and does not use a configured fallback.
PATH, HOME, LANG, LC_ALL and the names in pass_env.stderr is discarded.Content-Length, not wait for end of input.max_concurrent = N (N at least 1, valid only with command) to change it; a request over it gets 502.PATH unless it is absolute, so prefer an absolute path.expect_uid, Reiki reads the peer’s uid after connecting, before writing anything, and refuses with 502 on a mismatch.expect_uid (see security). In a rootless container or user namespace, the uid is the mapped one.expect_uid is set: without both, whoever can write the directory could swap the socket and receive every credential. Sticky directories such as /tmp stay allowed.reiki check and startup warn when the socket is missing or is not a socket, and about a sticky group- or world-writable directory without expect_uid, even when the socket does not exist yet.curl --unix-socket PATH http://localhost/....args, pass_env and max_concurrent need command; expect_uid needs socket.http2 or title_case_headers; startup rejects both. It can serve an intercepted tunnel host: Reiki terminates the client’s TLS and swaps the key, then hands the request to the forwarder.streaming, the 120-second request timeout bounds the whole exchange: connect, send and the full response; with it, only the wait for the head. response_idle_timeout_secs and response_timeout_secs also bound the body.[upstream.openai]
base_url = "https://api.openai.com/v1"
Client base URL: http://127.0.0.1:4097/openai
Covered by tests with invalid keys: a 401 takes the key out of rotation and moves the request to the next key. A 429 whose body names a quota, such as insufficient_quota, cools the key until its quota_resets instant when one is set, else for the Retry-After the response carries, else for 5 h. Other 429s cool it for Retry-After, or 60 s without one. These rules are unit-tested but not exercised with a real exhausted key.
[upstream.zen]
base_url = "https://opencode.ai/zen/v1"
Client base URL: http://127.0.0.1:4097/zen. For OpenCode, set provider.opencode.options.baseURL to that URL and apiKey to the stream token.
Some providers answer a usable key with a transient quota 429 (FreeUsageLimitError) and accepts the next request. Set confirm_quota = true to send such a 429 once more on the same key after a 3-second pause before cooling it; only a repeated quota 429 takes the key out of rotation. A genuinely exhausted key pays one extra request and 3 seconds per request.
A free-tier account answers a paid model with 402 while its free models keep working, so a whole-key cooldown would take them down too. A 402 and a plain rate-limit 429 are scoped to the request’s route and model: the key keeps serving other combinations, and /status lists the cooling combination under scopes. A quota 429 still cools the whole key, because it concerns the account, not one model. scope_escalate_after (default 3, 2 to 16) sets how many models may be cooling before another failure cools the key.
Zen’s free models only accept requests carrying OpenCode’s own headers; Reiki forwards every header except the credential, so the free tier keeps working.
A 403 “Model access is disabled” can apply to one endpoint only (OpenCode probes /responses before /chat/completions). Reiki tries the next key for that request, leaves the key in rotation, and returns the upstream’s answer if no key helps.
[upstream.anthropic]
base_url = "https://api.anthropic.com"
auth_header = "x-api-key"
auth_format = "{key}"
Client base URL: http://127.0.0.1:4097/anthropic. The SDK adds /v1/messages and its anthropic-version header, and Reiki passes both through.
A 400 “credit balance is too low” counts as quota exhaustion, from Anthropic’s documented error body.
[upstream.gemini]
base_url = "https://generativelanguage.googleapis.com"
auth_header = "x-goog-api-key"
auth_format = "{key}"
Client base URL: http://127.0.0.1:4097/gemini. The Gemini API reports an invalid key as a 400 with API_KEY_INVALID; Reiki treats that as a rejected key. A client that sends the key as a ?key= query parameter is accepted when no credential header is present, and Reiki removes the parameter before forwarding.
Covered by tests with an invalid key, including a client authenticating with x-goog-api-key.
Any HTTP API that takes its key in a header can be an upstream. For OpenAI-compatible services such as OpenRouter, Groq or Together, use the OpenAI block with the service’s base URL:
[upstream.openrouter]
base_url = "https://openrouter.ai/api/v1"
Reiki decides a key failure from the status code, the Retry-After header and a few error-body phrases. A provider that signals quota or key problems differently may need a new rule in src/classify.rs. These services are untested.
A 401 takes the key out of rotation at once, and one request disables at most one key: a second key that refuses the same request is tried but left alone. A request that every key refuses (a malformed credential header, a model the account cannot use) therefore still costs one good key. Set hold_unauthorized = true to hold that disable back until a later key serves the same request; only then is the first refusing key disabled. If every key of the upstream refuses, or the request ends in a fallback upstream or an error, no key is disabled. The default is false. While a bad key waits for a request to succeed, each request that reaches it pays one extra attempt.
Reiki reads a 401, a 403 or a quota phrase as a key failure and tries the next key. A 401, or a 403 whose body names an invalid, expired or revoked credential, disables the key; any other 403 skips the key for that request only. That is wrong for an upstream where those statuses mean something else, such as a git server that answers 403 to a read-only token that tries to push. Set pass_errors = true to relay every error status to the client as it is. No key is cooled, disabled or replayed, and the key’s failover and fallback do not apply. An error body is relayed up to 64 KiB, scrubbed of the upstream key first; a longer body is cut there and the response carries x-reiki-truncated: 1.
An upstream can name one other upstream to use when none of its keys is available:
[upstream.zen.fallback]
upstream = "openrouter"
model_aliases = { "*" = "openrouter/free" }
When every zen key in the stream’s chain is cooling or has just failed, Reiki sends the same request to openrouter instead of answering 503; /zen/chat/completions becomes /chat/completions on the target. Later requests skip the cooling keys and go straight to the target until they recover.
fallback is not followed.model_aliases, then the target’s own model_aliases and drop_params. A "*" key matches any model without an entry of its own, which swaps a provider-specific name for one the target serves; without a mapping the name is unchanged.falling back warning and the target upstream’s reiki_requests_total count them.An upstream with passthrough = true makes Reiki a plain reverse proxy for it, like an nginx or Traefik egress:
[upstream.zen-direct]
base_url = "https://opencode.ai/zen/v1"
passthrough = true
http2 = true
forward_accept_encoding = true
A request to /zen-direct/chat/completions goes to https://opencode.ai/zen/v1/chat/completions with the client’s headers, Authorization and x-api-key included, and its query string unchanged, including any key= parameter. Plugins may rewrite the body. Whatever comes back is returned as it is, errors too.
fallback.-.Host and Content-Length are dropped. Accept-Encoding is forwarded when the upstream sets forward_accept_encoding, and replaced with identity otherwise.