(MOT-4014) fix: align context compaction with router output budget

[ytallo]

Problem

Context assembly reserves the model catalog's maximum output ceiling even when the router will request a smaller effective output allocation. This unnecessarily reduces the usable input budget and can trigger premature compaction for any provider or model whose effective router allocation differs from its catalog ceiling.

Solution

• add the provider-agnostic router::models::budget contract, backed by the same output-token precedence used by router::chat
• resolve context-manager limits from the router's effective output allocation, with compatibility fallback during rolling upgrades
• expose full initial size, summarized-head size, assembled size, usable input, and effective output allocation in compaction records
• update schemas and architecture documentation for the new contract

Impact

Sessions across registered providers and models can use their available input capacity without premature compaction while retaining the existing hard over-budget protection. The effective budget accounts for provider configuration, model ceilings, provider defaults, router limits, and model-specific input limits.

Validation

• context-manager: unit, BDD, integration, manifest, and schema tests
• llm-router: unit, live-engine integration, and schema tests
• harness: unit, manifest, and schema tests
• formatting and diff validation

Refs MOT-4014

Summary by CodeRabbit

• New Features

  • Added model budget resolution, including effective output-token limits.
  • Context assembly responses now report effective output limits and additional token-count details.
  • Compaction telemetry now includes expanded token-budget measurements.
  • Added permission support for model budget inspection.

• Bug Fixes

  • Model budgeting now respects router-configured output caps.
  • Improved compatibility when token-accounting fields are missing from older responses.

• Documentation

  • Updated architecture and integration documentation to reflect the new model budget resolution flow.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
workers	Ready	Preview, Comment	Jul 15, 2026 11:16am
workers-tech-spec	Ready	Preview, Comment	Jul 15, 2026 11:16am

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: a7759f3f-cd64-47e2-9f91-383251fda685

📥 Commits

Reviewing files that changed from the base of the PR and between af03d2d and b332791.

📒 Files selected for processing (25)

• context-manager/architecture/README.md
• context-manager/architecture/integration.md
• context-manager/architecture/internals.md
• context-manager/src/adapters/router.rs
• context-manager/src/core/budget.rs
• context-manager/src/functions/assemble.rs
• context-manager/src/functions/mod.rs
• context-manager/src/main.rs
• context-manager/src/ports.rs
• context-manager/src/types.rs
• context-manager/tests/common/fakes.rs
• context-manager/tests/golden/schemas/context.assemble.json
• context-manager/tests/golden/schemas/context.compact.json
• context-manager/tests/golden/schemas/context.count-tokens.json
• context-manager/tests/golden/schemas/context.prune.json
• harness/src/clients/context.rs
• harness/src/turn_loop.rs
• iii-permissions.yaml
• llm-router/src/catalog/handlers.rs
• llm-router/src/register.rs
• llm-router/src/surface.rs
• llm-router/src/types/router.rs
• llm-router/tests/golden/schemas/router.models.budget.json
• llm-router/tests/integration.rs
• llm-router/tests/schemas.rs

---

📝 Walkthrough

Walkthrough

Changes

The router gains router::models::budget, returning a resolved model with its effective output-token limit. Context-manager consumes this budget, clamps resolved limits, exposes token accounting in assembly responses, and persists compaction telemetry. Documentation, schemas, permissions, adapters, fakes, and tests are updated accordingly.

Model budget flow

Layer / File(s)	Summary
Router budget endpoint llm-router/src/types/router.rs, llm-router/src/catalog/..., llm-router/src/surface.rs, llm-router/src/register.rs, llm-router/tests/..., iii-permissions.yaml	Adds and registers the typed budget endpoint, computes effective limits, updates schemas and registration tests, and permits the function.
Context budget resolution context-manager/src/ports.rs, context-manager/src/adapters/router.rs, context-manager/src/functions/mod.rs, context-manager/src/core/budget.rs, context-manager/tests/common/fakes.rs	Threads ModelBudget through the resolver contract and adapter, applies effective output caps, and updates the fake resolver and unit test.
Assembly token telemetry context-manager/src/functions/assemble.rs, context-manager/tests/golden/schemas/context.assemble.json, harness/src/clients/context.rs, harness/src/turn_loop.rs	Adds token-accounting response fields, tolerant deserialization, schema requirements, and persisted compaction telemetry.
Documentation and contract alignment context-manager/architecture/..., context-manager/src/main.rs, context-manager/src/types.rs, context-manager/tests/golden/schemas/context.*.json	Replaces documented model-limit references with router::models::budget.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant ContextAssembly
  participant ModelResolver
  participant Router
  participant BudgetCalculator
  ContextAssembly->>ModelResolver: resolve model and limits
  ModelResolver->>Router: router::models::budget
  Router->>BudgetCalculator: resolve effective max output tokens
  BudgetCalculator-->>Router: effective limit
  Router-->>ModelResolver: model and effective limit
  ModelResolver-->>ContextAssembly: resolved model budget
  ContextAssembly-->>ContextAssembly: assemble and report token accounting

Loading

Possibly related PRs

• iii-hq/workers#474: Adds the internal metadata pattern used when registering the new router endpoint.
• iii-hq/workers#495: Also updates context assembly budgeting and token-accounting contracts.

Suggested reviewers: sergiofilhowz, andersonleal

Poem

I’m a rabbit with budgets tucked tight,
Routing model limits just right.
Tokens count, tails compact,
Schemas keep the contract intact.
Hop, hop—effective caps take flight!

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/effective-context-budget

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

skill-check — worker

0 verified, 42 skipped (no docs/).

Layer	Result
structure	✓
vale	✓
ai	✓
render	✓

Four for four. Nicely done.