feat: experimental OpenAI to Anthropic cross-protocol translation #2
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "trevin/feat/openai-anthropic-translation"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
Adds experimental OpenAI→Anthropic cross-protocol translation so that OpenAI-only clients (which discover models via
GET /v1/modelsand POST/v1/chat/completionsfor everything) can reach Anthropic-served models without speaking the native Anthropic protocol.Native protocol passthrough remains the recommended path — clients that speak the provider's native protocol (like the opencode aor plugin, which defers to the correct API per model) get verbatim passthrough with no transformation. Translation is per-attempt: a model served by a mixed-protocol provider tier may passthrough on one attempt and translate on the next. This also enables cross-protocol fallback (previously broken) — an OpenAI provider's 5xx can fall back to an Anthropic provider that translates on the fly.
Google translation is a planned extension point (no date). Images/vision are deferred (raises a clear 400
aor_translate_error).What's supported (v1)
input_json_deltapartialsreasoning_effort(low/medium/high) → Anthropicthinking.budget_tokensvia a heuristic table; explicitthinkingdicts pass throughchat.completion.chunkSSE, includingstream_options.include_usageserver_error)Architecture
Translation is a separate
aor/translate/module (not an extension of theAdapterProtocol). Adapters stay body-shape-agnostic (model id + auth + path + error-classify only). TheTranslatorProtocol + registry is a clean extension point for future Google support — a new module registered for("openai", "google")would plug in without touching the proxy.The proxy detects the client's protocol from the request path (
client_protocol()helper inadapters/__init__.py) and the serving provider's protocol from config (apifield). When they differ and a translator is registered, the request body is translated before forwarding and the response is translated back. All sticky/backoff/usage bookkeeping is preserved unchanged — translation is orthogonal to routing.Commits
bfcc344— translate/ package + OpenAI→Anthropic request translatoracdabe4— Anthropic response + error translation014300b— Anthropic stream translation (SSE → OpenAI chunks)b4309cc— proxy.py integration (per-attempt translator in buffered + streaming)9a03e15— proxy integration tests (cross-protocol fallback, streaming, errors)f8820b9— README docs for experimental translationTesting
uv run ruff check .— cleanuv run ruff format --check .— cleanuv run pytest -q— 219 passed (135 baseline + 73 translator unit tests + 11 proxy integration tests)Test coverage
tests/test_translate_anthropic.py): registry, path suffix, request translation (system lifting, max_tokens injection, tools, tool_choice, reasoning, dropped fields, error cases), response translation (text, tool_use, stop_reason mapping, usage), error translation (status/type maps, malformed bodies), stream translation (message_start, content_block_start tool_use stub, text_delta, input_json_delta partials, message_delta stop_reason, message_stop finalize, mid-stream error, include_usage, ping/unknown).tests/test_proxy.py): basic text round-trip, tools round-trip, streaming text, streaming tools, streaming include_usage, cross-protocol fallback (openai 5xx → anthropic translates, 2 usage rows), error 429→OpenAI shape, error 529→503, error 400 non-retryable, image_url request error, native Anthropic passthrough with translator registered.Known nuance
For translated errors, the usage row's
http_statusrecords the translated status (e.g. 503 for an Anthropic 529) rather than the original upstream status. Theerror_kindthat drives backoff is correctly computed on the original native body, so routing decisions are unaffected — this is purely an admin-UI observability detail.translate_response converts a buffered Anthropic /messages body to an OpenAI chat.completion object: text blocks join into message.content, tool_use blocks become tool_calls (input dict -> arguments JSON string), thinking blocks are dropped (no standard OpenAI home in v1), stop_reason maps to finish_reason (end_turn/stop_sequence->stop, max_tokens->length, tool_use->tool_calls, refusal->content_filter, default stop), and usage remaps input_tokens/output_tokens -> prompt_tokens/completion_tokens with cache_* fields dropped. translate_error converts Anthropic's {type:error, error:{type,message}} shape to OpenAI's {error:{message,type,param,code}}. Notable remaps: 529 (Anthropic "overloaded") -> 503 with type server_error; most other statuses and types pass through. Malformed upstream error bodies yield a generic server_error instead of crashing. 26 new unit tests cover every stop_reason, text/tool/thinking blocks, usage remapping, id synthesis, and each error status path. The stream method remains stubbed for the next commit.translate_stream is a stateful async generator that consumes Anthropic SSE lines and yields OpenAI chat.completion.chunk SSE bytes: - message_start: capture id/model/input_tokens, emit the initial {role:"assistant"} delta chunk (once). - content_block_start (tool_use): emit a tool_calls stub delta with id/name and an empty arguments string. A sequential tool_ordinal is tracked separately from Anthropic's block index (which also counts text blocks) so OpenAI tool_calls[].index starts at 0. - content_block_delta text_delta: emit a content delta. - content_block_delta input_json_delta: stream the partial_json string through as tool_calls[].function.arguments partials (the client reassembles them into valid JSON). - thinking_delta: dropped (no OpenAI home in v1). - message_delta: capture stop_reason -> finish_reason and output_tokens. - message_stop: finalize emits the terminal finish chunk + optional usage chunk (when include_usage) + [DONE]. - ping/unknown: ignored. - mid-stream error: emits an OpenAI-shaped error event chunk then [DONE] and terminates (consistent with the proxy's no-mid-stream- retry rule; the _ERROR_TYPE_MAP is applied so overloaded_error -> server_error). An abrupt upstream end without message_stop is finalized defensively with a default "stop" finish + [DONE]. The proxy will pass resp.aiter_lines() through this generator and set media_type=text/event-stream; the background-task close semantics are preserved by the proxy integration in a later commit. 37 new unit tests cover text/tool/thinking/usage/ping/error/abrupt-end paths. Test SSE fixtures are built via json.dumps helpers to avoid manual escape errors. The full suite is now 208 passing.Wire the experimental OpenAI→Anthropic translator into the proxy's per-attempt loop. Translation activates only when the client protocol (diffed from the request path) differs from the serving provider's declared API — when the translator is None the proxy's verbatim passthrough behaviour is unchanged. Per-attempt selection matters because a model's provider tier can mix protocols: one request may passthrough attempt 1 (OpenAI provider) and translate attempt 2 (Anthropic provider). This also enables previously broken cross-protocol fallback as a side benefit. Changes: - adapters/__init__.py: add client_protocol(path) helper, reusing the existing _GOOGLE_PATH_RE to classify /messages (anthropic), /models/{id}:… (google), or everything else (openai). - proxy.py: compute client_proto + include_usage before the attempt loop; select translator = get_translator(client_proto, prov.api) per attempt. When translating, translator.translate_request replaces adapter.rewrite_request and translator.upstream_path_suffix replaces adapter.rewrite_path; upstream headers still come from the adapter. Buffered 2xx: translate_response → json.dumps; non-2xx: classify retryability on the ORIGINAL native body (drives the walk decision), then translate_error for client surfacing (new status + body + content-type application/json). Streaming 2xx: _safe_translate_stream wraps the translator's async generator so a mid-stream TranslateError yields an OpenAI error chunk + [DONE] instead of a dropped connection (honours the existing "commit at first byte, never retry mid-stream" rule). Non-2xx: same translate-error pattern as buffered. TranslateError on request → 400 (deterministic, no retry). TranslateError or ValueError on response translation → 502 _translate_fail_outcome (non-retryable). All sticky/backoff/usage bookkeeping is preserved unchanged — translation is purely orthogonal to routing.Three minor fixes from PR review: 1. max_tokens: 0 was silently coerced to the 4096 default because the guard used truthiness (`if max_tokens:`) instead of `is not None`. An explicit 0 is now preserved. Same applies to max_completion_tokens. 2. The `created` field in translated responses and stream chunks was hardcoded to 0 (1970-01-01). Now uses `int(time.time())` as expected for the OpenAI unix-timestamp field. 3. The Anthropic message id (e.g. `msg_01…`) was reused verbatim as the OpenAI response/chunk id. Now wrapped as `chatcmpl-aor-{native_id}` so the id carries the `chatcmpl-` prefix that some OpenAI clients expect. Falls back to `chatcmpl-aor` when the upstream sends no id.Addressed all four review findings in two follow-up commits (
e6ba08a,bb37d75). 221 tests passing, ruff clean.Medium — request TranslateError bailed the entire walk (
e6ba08a)The real bug. A
TranslateErrorfromtranslate_request(e.g.image_urlinput, which is deferred) used toreturna 400 immediately, skipping all remaining providers. In a mixed-protocol tier — the explicit design goal — a later passthrough provider (whoseapimatches the client protocol) could have served the request natively. The bug was also nondeterministic per client key because consistent-hash ordering determines which provider is tried first within a tier.Fix: the failed attempt now
continues the walk (no upstream call → no usage row, no retry-budget consumption). A loop-exhaustion safety net surfaces the 400 via_surface(last_error)if every attempt fails for translation reasons.New test:
test_translate_skip_falls_through_to_passthrough— a mixed-tier config (anthropic priority 1, openai priority 2, both servingmix) confirms animage_urlrequest skips the anthropic translator and succeeds via the openai passthrough, with exactly one usage row (the openai attempt only).Low — three polish fixes (
bb37d75)max_tokens: 0was silently coerced to 4096 (if max_tokens:→if max_tokens is not None:). Explicit 0 is now preserved.createdtimestamp was hardcoded to 0 (1970-01-01); now usesint(time.time()).chatcmpl-aor-{native_id}instead of reusingmsg_01…verbatim, so thechatcmpl-prefix is present for OpenAI clients that expect it.