Actually Open Router

A self-hosted, OpenAI-compatible AI API router. You run it on your own machine, point your existing OpenAI-compatible clients at it, and it routes requests across your configured upstream providers — with per-client cache stickiness and automatic fallback before any bytes are streamed.

Not a SaaS. There's no hosted version. It's a single Python process, a sqlite database, and a TOML config file.

Features

• OpenAI-compatible surface — drop-in base URL for any client that speaks OpenAI (/v1/chat/completions, /v1/completions, /v1/embeddings, /v1/responses, GET /v1/models).
• Multi-protocol adapters — routes OpenAI-compatible, Anthropic (/v1/messages), and Google (/v1/models/{id}:generateContent) endpoints. The same downstream key works regardless of which auth header the client SDK uses (Authorization: Bearer, x-api-key, or x-goog-api-key).
• Experimental cross-protocol translation — OpenAI-only clients can reach Anthropic-served models via automatic request/response/stream translation. Native protocol passthrough remains recommended; see Experimental cross-protocol translation.
• Declarative TOML config — one file describes your providers, keys, and the model catalog. Validated once at startup; restart to apply changes.
• Cache-sticky routing — within a priority tier, a client API key is consistently hashed to the same provider so upstream prompt caches stay warm.
• Fallback before first byte — on a retryable upstream error (rate limit, quota, 5xx, network), the router tries the next candidate before any response bytes reach the client. Once streaming starts, errors pass through verbatim — no mid-stream retries.
• Downstream key management — mint and revoke sk-aor-… keys from a small admin UI. Keys are stored hashed; the full key is shown exactly once.
• Per-key model scope — restrict a downstream key to an allowlist of catalog models. The proxy enforces it (403 aor_model_scoped) and GET /v1/models reflects it per key. See docs/requirements-model-scope.md.
• Admin login lockout — per-IP brute-force lockout on the admin login (429 + Retry-After + banner), tunable via AOR_LOGIN_* env vars. The admin password is hashed at boot with PBKDF2-HMAC-SHA256 (~600k iterations). See docs/requirements-admin-login-hardening.md.
• Per-attempt usage logging — every upstream attempt is logged to sqlite (one row per attempt, not per request) for debugging routing decisions. No request bodies or PII are logged.
• Adapter-based — a small Adapter protocol governs model extraction, request/path rewriting, auth header construction, and error classification. Ships openai, anthropic, and google adapters.
• Upstream model discovery — at startup and on demand from the admin UI, fetches each provider's model list to surface available models and validate the config catalog (warns if a configured model isn't found upstream).

Quick start

Requirements: Python 3.14+ and uv.

git clone <this repo> actually-open-router
cd actually-open-router
uv sync

# 1. Create a config from the example
cp config.example.toml config.toml

# 2. Set the upstream API key env vars your config references, e.g.
export OPENCODE_GO_KEY_1=sk-...
export OPENCODE_GO_KEY_2=sk-...
export OPENCODE_GO_KEY_3=sk-...
export OPENCODE_ZEN_KEY=sk-...

# 3. Set an admin password (or leave unset to disable /admin/* with a 503)
export AOR_ADMIN_PASSWORD=changeme

# 4. Run
uv run aor
# → listening on http://127.0.0.1:8000

Point any OpenAI-compatible client at http://127.0.0.1:8000/v1 using a downstream key you mint via the admin UI (/admin).

To use the installed entrypoint instead of uv run:

aor          # uses config.toml in the cwd by default

Configuration

The full schema is described in docs/requirements.md. A commented reference config lives at config.example.toml.

The shape, in short:

[routing]
auto_fallback = true                 # global default; per-model may override

[providers.my-provider]
api               = "openai"         # adapter: "openai", "anthropic", or "google"
base_url          = "https://api.example.com/v1"   # must include /v1
provider_priority = 1               # lower = tried first (default 50)
model_prefix      = "myprefix"      # optional; resolves native_id -> prefix/id

  [[providers.my-provider.api_keys]]
  env = "MY_PROVIDER_KEY"           # NAME of an env var, not the value
  # key_priority = 50               # within a provider, lower first

  [[providers.my-provider.models]]
  id = "glm-5.2"                    # canonical, client-facing model id
  # native_id = "explicit/name"     # overrides prefix/id resolution

# Model catalog: a model is callable iff it has a [models.<id>] entry AND at
# least one provider declares it. Per-model provider overrides use inline
# tables.
[models."glm-5.2"]
  [models."glm-5.2".providers.my-provider]
  priority = 3                      # re-rank this provider for this model only

TOML gotcha: a dot in a bare table key is parsed as nesting, so a model id that contains a dot must be quoted in the table header: [models."glm-5.2"]. The id value is a string and needs no special quoting.

Startup validation

The router refuses to boot if the config is invalid. Checks performed once at startup include: TOML parse errors, pydantic validation, a catalog model with no serving providers, an orphan provider model declaration (id not in the catalog), an unset/empty upstream key env var, an unknown adapter api (must be openai, anthropic, or google), a duplicate api_keys.env within a provider, and a per-model provider override that names a provider not serving that model.

Native id resolution

The id sent upstream is resolved as: explicit native_id → {model_prefix}/{id} if a prefix is set → the bare id otherwise. The client always sees and sends the canonical id.

Environment variables

Variable	Default	Meaning
AOR_CONFIG	./config.toml	Path to the TOML config.
AOR_DB_PATH	./data/aor.db	sqlite database path.
AOR_HOST	127.0.0.1	Bind host.
AOR_PORT	8000	Bind port.
AOR_ADMIN_PASSWORD	unset	Admin UI password. When unset, /admin/* returns 503 while /v1/* keeps serving.
AOR_ADMIN_SESSION_HOURS	168	Admin session lifetime in hours.
AOR_COOKIE_SECURE	unset	When true/1, set the Secure flag on the admin session cookie. Set when serving the admin UI over HTTPS only.
AOR_ADMIN_API_TOKEN	unset	Bearer token for the JSON admin API (/admin/api). When unset, /admin/api returns 503 while /v1/* and the browser admin keep serving.
AOR_LOGIN_MAX_FAILURES	5	Max failed admin login attempts per IP inside the window before the IP is locked.
AOR_LOGIN_WINDOW_SECONDS	900	Admin login failure-counting window in seconds.
AOR_LOGIN_LOCKOUT_SECONDS	900	How long an IP stays locked after the failure limit.
AOR_TRUST_X_FORWARDED_FOR	unset	When true/1, use the leftmost X-Forwarded-For entry as the client IP for the admin login lockout. Enable only behind a trusted proxy that overwrites the header on every request — otherwise clients can spoof any source IP and bypass the lockout.
one per declared upstream key	—	Each [[providers.*.api_keys]] env = "X" reads the upstream key from X.

Routing

For each request:

1. Look up the body's model in the catalog — unknown models return 404.
2. Gather the providers serving it and rank by effective priority (a per-model override beats the provider's provider_priority; lower first).
3. Group providers into tiers of equal priority. Within a tier, pick the provider by consistent_hash(client_key, model, provider) (highest wins) so the same client sticks to the same provider and keeps upstream prompt caches warm. Within a provider, keys are ordered by key_priority ascending, with ties broken by the consistent hash.
4. Forward via the adapter (rewrite body.model to the native id, rewrite the path for Google, replace the auth header with the upstream key). Non-streaming responses are buffered fully before return so a retryable failure can fall back; streaming commits at first byte.
5. On a retryable error and no bytes streamed yet and auto_fallback enabled and more candidates remain, advance to the next candidate. When candidates are exhausted, the last upstream error is surfaced verbatim. Mid-stream errors are passed straight through.

Retryable error classification (OpenAI adapter): 429 → rate limit; 402 → quota; 403 → quota only when the body error type indicates a quota/ usage-limit condition, otherwise non-retryable auth; 5xx → server error; network/timeout → network error. Other 4xx (including 401) are non-retryable.

Experimental cross-protocol translation

aor is a same-protocol passthrough by design: an OpenAI client request is forwarded to an OpenAI-served upstream, an Anthropic client request to an Anthropic-served upstream. This is the recommended way to use aor — clients that speak the provider's native protocol (like the opencode aor plugin, which defers to the correct API per model) get verbatim passthrough with no transformation.

For clients that only speak OpenAI (e.g. they discover models via GET /v1/models and POST /v1/chat/completions for everything), aor ships an experimental translation layer that lets those clients reach models served behind a different native protocol. v1 supports OpenAI→Anthropic; Google is a planned extension point.

How it works

The proxy detects the client's protocol from the request path (/v1/chat/completions → openai, /v1/messages → anthropic, /v1/models/{id}:… → google) and the serving provider's protocol from the config (api field). When they differ and a translator is registered for that pair, the request body is translated to the native protocol before forwarding, and the response is translated back to the client's protocol.

Translation is per-attempt: a model served by a mixed-protocol provider tier may passthrough on one attempt and translate on the next. This also enables cross-protocol fallback (previously broken) — an OpenAI provider's 5xx can fall back to an Anthropic provider that translates on the fly.

What's supported (v1)

• Text — string and block-list content
• Tool/function calling — tools, tool choices, tool results, and streaming input_json_delta partials
• Reasoning/thinking — reasoning_effort (low/medium/high) is mapped to Anthropic thinking.budget_tokens via a heuristic table; explicit thinking dicts pass through
• Streaming — Anthropic SSE events are translated to OpenAI chat.completion.chunk SSE, including stream_options.include_usage
• Error translation — Anthropic error bodies are reshaped to OpenAI error format (e.g. 529 "overloaded" → 503 server_error)

What's not supported

• Google — planned, no date. Google-served models require a native Google client (/v1/models/{id}:generateContent).
• Image/vision input — image_url content blocks raise a clear 400 error (aor_translate_error). Planned, no date.
• Native non-OpenAI clients reaching a mismatched-protocol upstream — an Anthropic client hitting a Google-served model stays passthrough (and will break). Use the matching client protocol instead.
• /v1/responses — the OpenAI Responses API is a different sub-shape than /chat/completions; aor does not translate between them.
• max_tokens — Anthropic requires it on every request; OpenAI treats it as optional. When the OpenAI client omits it, aor injects a default of 4096. Send max_tokens explicitly if you need a different value.

When translation fails

• Request translation error (e.g. image input) → 400 aor_translate_error (no upstream call is made).
• Response translation error (malformed upstream body) → 502 aor_translate_error.
• Mid-stream translation error → the client receives an OpenAI-shaped error event chunk followed by [DONE] (no mid-stream retry, per the existing streaming rule).

Admin UI

Browse to /admin (redirects to /admin/login). Log in with AOR_ADMIN_PASSWORD. From there you can generate new downstream keys (with an optional friendly label, shown in full exactly once), list existing keys by their label and identifying prefix, and revoke keys. The /admin/models page shows each provider's discovered upstream model list and config validation warnings (catalog models not found upstream); use the Refresh button to re-fetch on demand. The /admin/stats page shows at-a-glance provider health (a colored dot combining discovery reachability with recent error rate), per-provider usage counts with a CSS error-distribution bar, fallback counts, average latency, a per-model summary, and a table of the 50 most recent attempts (including which upstream key was used). Click a provider name for a per-upstream-key breakdown. Use the window selector (1h / 24h / 7d / All) to scope the aggregates. Upstream keys may also be given a friendly name in the config ([[providers.*.api_keys]] name field); the admin UI shows it instead of the raw env var name. There is intentionally no user table, no per-key metadata, and no rotation API in v1 — rotate by revoking and creating.

If AOR_ADMIN_PASSWORD is not set, the admin UI is disabled (503) while the proxy keeps serving /v1/*.

The browser UI is complemented by a JSON admin API at /admin/api for scripted key, budget, limits, and usage management (bearer token AOR_ADMIN_API_TOKEN) — see docs/requirements-admin-api.md. Per-key rate and concurrency limits are specified in docs/requirements-rate-concurrency-limits.md.

Observability

One sqlite row per upstream attempt in the usage table: ts, client_key_hash, canonical_model, provider, upstream_key_env, http_status, latency_ms, fallback_attempt, error_kind. A client request that falls back writes one row per attempt. No request bodies or header values are logged.

Development

uv sync                            # install dev extras
uv run ruff check .                # lint
uv run ruff format --check .       # format check
uv run pytest -q                   # tests

Docker deployment

An example Dockerfile and compose.yml are included for a single-container homelab deploy. The compose file clones and builds from this repo's default branch, so no local clone is needed on the deploy host.

1. On the deploy host, create an empty dir and fetch the compose file:

   mkdir aor-deploy && cd aor-deploy
   curl -O https://git.trevhost.com/devtrev/actually-open-router/raw/branch/dev/compose.yml
   

2. Create .env from .env.example and fill in your upstream keys and admin password.
3. Create config and data dirs:

   mkdir config data
   cp config.example.toml config/config.toml   # edit to taste
   chown -R 1000:1000 data                      # container runs as uid 1000
   

4. Build and start:

   docker compose up -d --build
   

5. Point your reverse proxy at 127.0.0.1:8000 — the port is bound to loopback only, so the container is not directly internet-reachable.

Set AOR_COOKIE_SECURE=true and serve /admin over HTTPS. Don't expose /admin on the public path without additional protection (login is protected by a per-IP brute-force lockout, but that is per-IP and in-memory only); keep it behind your reverse proxy's access control or on a LAN/Tailscale-only route.

Scope of v1

This is a small, self-hosted router by design. Out of scope: multi-host deployments, hot config reload, model aliasing, a key-management CLI, and non-OpenAI/Anthropic/Google adapters. See docs/requirements.md for the full spec.

LICENSE

See LICENSE. Actually Open Router is free software available according to the terms of AGPL-3.0.

Actually Open Router IS IN NO WAY AFFILIATED WITH OpenRouter OR ANY OF ITS RELATED SOFTWARE OR COMPONENTS.

CONTRIBUTING

See devtrev/CONTRIBUTING