feat: model limit config + /v1/models cost+limit metadata #5

Merged

devtrev merged 1 commit from trevin/feat/model-limits-cost-metadata into dev

2026-07-10 23:58:00 -06:00

devtrev commented

2026-07-10 23:19:53 -06:00

Owner

Summary

Add per-model limit config (context/input/output token limits) with per-provider override support, mirroring the existing pricing resolution chain (provider override → model default → None).
Extend GET /v1/models to expose cost and limit as additive non-standard fields, sourced from the primary (highest-priority) provider's resolved values.
This lets downstream clients (e.g. the opencode-aor plugin) populate cost-tracking and context-window displays without a separate catalog like models.dev.

Changes

aor/config.py: LimitConfig pydantic model, added to ModelConfig, ModelProviderOverride, CandidateProvider, and ResolvedModel. Resolution: provider override → model default → None.
aor/models_endpoint.py: cost and limit fields exposed in /v1/models response (omitted when not set).
config.example.toml: limit examples at model level + per-provider override.
docs/requirements-key-budgets.md: new section documenting limit config, /v1/models metadata exposure, and when to split models with greatly varying limits/pricing into separate catalog entries.
tests/test_config.py: 6 new tests for limit config resolution.
tests/test_proxy.py: 1 new test for /v1/models cost+limit exposure.

When to split models

When limits or pricing vary greatly across providers for the same model, consider splitting into separate catalog models so downstream clients get deterministic metadata per model selection. See docs/requirements-key-budgets.md for details.

Testing

uv run ruff check .          # All checks passed
uv run ruff format --check . # 38 files already formatted
uv run pytest -q             # 291 passed

Depends on

PR #4 (trevin/feat/key-budgets-spend-visibility) — this branch is based on it since pricing config and PricingConfig live there.

## Summary - Add per-model `limit` config (`context`/`input`/`output` token limits) with per-provider override support, mirroring the existing `pricing` resolution chain (provider override → model default → `None`). - Extend `GET /v1/models` to expose `cost` and `limit` as additive non-standard fields, sourced from the primary (highest-priority) provider's resolved values. - This lets downstream clients (e.g. the `opencode-aor` plugin) populate cost-tracking and context-window displays without a separate catalog like models.dev. ## Changes - `aor/config.py`: `LimitConfig` pydantic model, added to `ModelConfig`, `ModelProviderOverride`, `CandidateProvider`, and `ResolvedModel`. Resolution: provider override → model default → `None`. - `aor/models_endpoint.py`: `cost` and `limit` fields exposed in `/v1/models` response (omitted when not set). - `config.example.toml`: limit examples at model level + per-provider override. - `docs/requirements-key-budgets.md`: new section documenting limit config, `/v1/models` metadata exposure, and when to split models with greatly varying limits/pricing into separate catalog entries. - `tests/test_config.py`: 6 new tests for limit config resolution. - `tests/test_proxy.py`: 1 new test for `/v1/models` cost+limit exposure. ## When to split models When limits or pricing vary **greatly** across providers for the same model, consider splitting into separate catalog models so downstream clients get deterministic metadata per model selection. See `docs/requirements-key-budgets.md` for details. ## Testing ``` uv run ruff check . # All checks passed uv run ruff format --check . # 38 files already formatted uv run pytest -q # 291 passed ``` ## Depends on PR #4 (`trevin/feat/key-budgets-spend-visibility`) — this branch is based on it since `pricing` config and `PricingConfig` live there.

devtrev added 1 commit

2026-07-10 23:19:53 -06:00

feat: model limit config + /v1/models cost+limit metadata dc9bd65ff0

Add per-model `limit` config (context/input/output token limits) with
per-provider override support, mirroring the existing `pricing` resolution
chain (provider override → model default → None).

Extend GET /v1/models to expose `cost` and `limit` as additive non-standard
fields, sourced from the primary (highest-priority) provider's resolved
values. This lets downstream clients like the opencode-aor plugin populate
cost-tracking and context-window displays without a separate catalog
(models.dev submission).

When limits/pricing vary greatly across providers for the same model, the
docs recommend splitting into separate catalog models so clients get
deterministic metadata per model selection.

Tests: 6 new config tests (no-limit, model-level, with-input, provider
override, override-without-limit, explicit-free), 1 new /v1/models test
(cost+limit present, cost-only, limit-only, unpriced+unlimited). 291 pass.

devtrev merged commit 2f9a386cdf into dev

2026-07-10 23:58:00 -06:00

devtrev referenced this pull request from a commit

2026-07-10 23:58:01 -06:00

Merge pull request 'feat: model limit config + /v1/models cost+limit metadata' (#5) from trevin/feat/model-limits-cost-metadata into dev

devtrev referenced this pull request from a commit

2026-08-20 17:00:01 -06:00

fix(modelsdev): address review findings