feat(routing): provider-aware session affinity hints for prompt caching

[davidgovea]

What does this PR do?

Most inference providers now support some form of prompt prefix caching, but the cache-hint mechanism varies per provider: Fireworks uses an x-session-affinity header, OpenAI uses a prompt_cache_key body field, xAI has x-grok-conv-id, etc. There's no standard, and this is a real pain point across the ecosystem: several PRs and issues in other agentic clients are stalled or incomplete because each one hits the same "which provider am I actually talking to?" problem before it can send the right hint.

This PR introduces a data-driven routing table that maps (provider, api_mode) → request metadata, so each provider gets exactly the cache hint it understands and unknown routes get nothing. It also generalizes the existing hardcoded prompt_cache_key for OpenAI Codex into this same table, and threads session_id through all LLM call sites (primary, auxiliary, context compression, title generation, memory flush, iteration-limit summaries).

Related Issue

N/A — standalone improvement. See "Related ecosystem work" below for cross-project context.

Type of Change

Performance & robustness

Changes Made

Core routing logic:

• agent/model_metadata.py — new build_openai_request_session_routing() with a provider/api_mode lookup table, provider inference from base_url, and template rendering

Current routing entries:

Provider	API mode	Routing
fireworks	chat_completions	x-session-affinity: {session_id} header
openai / openai-codex	codex_responses	prompt_cache_key: {session_id}

Future / WIP — not yet wired, pending testing:

Provider	Likely mechanism	Notes
Cloudflare Workers AI	x-session-affinity header	Documented for prefix-cache locality
xAI	x-grok-conv-id header or prompt_cache_key	Supports both; needs testing to confirm which path works via OpenAI-compat
Mistral	x-session-affinity or similar	Anecdotal reports of prefix caching with affinity headers; no public docs yet

Integration points:

• run_agent.py — _apply_request_session_routing() method applied to primary _build_api_kwargs, memory flush, and iteration-limit summary calls
• agent/auxiliary_client.py — session_id param on call_llm/async_call_llm, routing applied in _build_call_kwargs, api_mode attribute on all auxiliary client wrappers, _CodexCompletionsAdapter forwards prompt_cache_key/extra_headers/extra_body
• agent/context_compressor.py — forwards session_id to auxiliary call_llm
• agent/title_generator.py — forwards session_id to auxiliary call_llm

How to Test

1. pytest tests/ -q — new tests pass, but there are prexisting failures in my current env (WIP)
2. Configure a Fireworks endpoint and inspect outgoing requests — x-session-affinity header should be present with the session ID
3. Use OpenAI Codex route — prompt_cache_key should still be set to session_id (unchanged behavior, now via the routing table)
4. Use GitHub Copilot route — no prompt_cache_key should be sent (unchanged behavior, copilot provider is not in the table)
5. Use a local/unknown endpoint — no routing metadata should be added

Provider docs

• Fireworks prompt caching
• Fireworks changelog — session affinity via user / x-session-affinity
• OpenAI prompt caching / prompt_cache_key
• Azure OpenAI prompt caching / prompt_cache_key
• xAI prompt caching / x-grok-conv-id / prompt_cache_key
• Cloudflare on x-session-affinity for prefix-cache locality

Related ecosystem work

These all circle around the same problem — agentic clients need provider-aware gating to send the right cache hint on the right route:

• OmniRoute #518 — preserving prompt_cache_key through Responses → Chat translation
• OpenCode #4317 — stable conversation IDs / provider cache-key hooks
• OpenClaw #16387 — provider-aware session cache hints
• Zed #36215 — why vendor-native cache hints need provider-aware gating

Checklist

Code

• I've read the Contributing Guide
• My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
• I searched for existing PRs to make sure this isn't a duplicate
• My PR contains only changes related to this fix/feature (no unrelated commits)
• I've run pytest tests/ -q and all tests pass (I'm seeing preexisting test failures on main: will continue to iterate)
• I've added tests for my changes (required for bug fixes, strongly encouraged for features)
• I've tested on my platform: macOS 15

Documentation & Housekeeping

• I've updated relevant documentation (README, docs/, docstrings) — or N/A
• I've updated cli-config.yaml.example if I added/changed config keys — or N/A
• I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
• I've considered cross-platform impact (Windows, macOS) per the compatibility
  guide — or N/A
• I've updated tool descriptions/schemas if I changed tool behavior — or N/A

[alt-glitch]

Thanks for this proposal — the idea was sound and main has since realized provider-aware session cache affinity in the transport layer: agent/transports/codex.py threads session_id into prompt_cache_key and the x-grok-conv-id header (~L253-374), with the same threading in chat_completions.py. Closing as superseded by the equivalent merged mechanism (verified @ f64e4f4).