Skip to content

[DO-NOT-MERGE-YET] complexity cache-aware session routing integration - #6043

Closed
Madhuvod wants to merge 1 commit into
08-11-_do-not-merge-yet_complexity_cache-aware_session_tier_policyfrom
08-11-_do-not-merge-yet_complexity_cache-aware_session_routing_integration
Closed

Madhuvod wants to merge 1 commit into
08-11-_do-not-merge-yet_complexity_cache-aware_session_tier_policyfrom
08-11-_do-not-merge-yet_complexity_cache-aware_session_routing_integration

Conversation

@Madhuvod

@Madhuvod Madhuvod commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Summary

Cache-aware session mode previously shared a single code path with pinned mode, which meant it could not correctly handle per-turn classification, atomic tier-change decisions, or the distinction between a held tier and a freshly classified one. This PR splits the two modes into separate implementations so each behaves correctly and independently.

Changes

  • Extracted classifyWithPinnedSession and classifyWithCacheAwareSession from the single withComplexitySession closure, replacing a mode-conditional branch inside one function with two dedicated paths dispatched by a switch on state.Config.Mode.
  • Cache-aware mode now classifies on every relevant turn, then calls store.Update atomically to decide whether the session may move. An unchanged decision aborts the replicated write and falls back to a coarse Get to slide the TTL, avoiding one write per request when the tier is stable.
  • Pinned mode retains its original behaviour: look up the stored tier, return it if found, classify and persist only on the first turn.
  • Replaced the two separate Mode and TTL fields on complexitySessionState with a single Config configstore.ComplexitySessionConfig snapshot, so both paths read from the same policy values captured at request start and cannot mix thresholds from a concurrent config reload.
  • Added publishCacheAwareSessionTier to log the policy decision (tier, proposed tier, switched flag, reason, identity) and clear the score from context when the held tier differs from the proposal, preventing a score calculated for one tier from being attributed to another.
  • Added persistInitialSessionTier as a shared helper used by both modes when writing the first record for a new session.
  • Added errSessionTierDecisionUnchanged sentinel so Update can abort without a replicated write while still allowing the caller to distinguish a no-op from a real error.
  • Added failingUpdateSessionStore test double and five new test cases covering: no replicated write on an unchanged cache-aware tier, escalation with semantic mechanism preserved, sustained-downgrade gating, hold when classification produces no tier, and fallback to the proposal when Update fails.
  • Updated schema descriptions, UI copy, and comments throughout to accurately describe what each mode does, particularly that cache-aware mode pays an embedding cost on every complexity-routed turn.

Type of change

  • Bug fix
  • Feature
  • Refactor
  • Documentation
  • Chore/CI

Affected areas

  • Core (Go)
  • Transports (HTTP)
  • Providers/Integrations
  • Plugins
  • UI (React)
  • Docs

How to test

# Core/Transports
go test ./plugins/governance/... -run TestWithComplexitySession
go test ./plugins/governance/... -run TestRecordSessionRouteObservation
go test ./...

# UI
cd ui
pnpm i || npm i
pnpm build || npm run build

Validate cache-aware behaviour end-to-end by configuring a session with mode: cache_aware, sending several turns with the same session identity, and confirming via routing engine logs that:

  • Every complexity-routed turn shows a Cache-aware complexity session log entry with switched=false when the tier is stable.
  • No replicated store write occurs on unchanged decisions (observable via store metrics or delegate counters in tests).
  • A turn whose classifier proposes a higher tier and meets the confidence threshold shows switched=true and the new tier takes effect immediately.
  • A downgrade proposal accumulates pending_turns until downgrade_after_n_turns is satisfied before switching.

Breaking changes

  • Yes
  • No

Related issues

Security considerations

No auth, secrets, PII, or sandboxing changes. Session keys remain tenant-namespaced and TTL-bounded as before.

Checklist

  • I read docs/contributing/README.md and followed the guidelines
  • I added/updated tests where appropriate
  • I updated documentation where needed
  • I verified builds succeed (Go and UI)
  • I verified the CI pipeline passes locally if applicable

@coderabbitai

coderabbitai Bot commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: f6ae78ed-e2c0-437d-a551-bc6a9697c3c8

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

Madhuvod commented Aug 11, 2026

Copy link
Copy Markdown
Contributor Author

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@Madhuvod
Madhuvod force-pushed the 08-11-_do-not-merge-yet_complexity_cache-aware_session_routing_integration branch from 82164fb to 3fbdd04 Compare August 11, 2026 12:48
@Madhuvod
Madhuvod force-pushed the 08-11-_do-not-merge-yet_complexity_cache-aware_session_tier_policy branch from 3b88289 to 23a3626 Compare August 11, 2026 12:48
@Madhuvod
Madhuvod force-pushed the 08-11-_do-not-merge-yet_complexity_cache-aware_session_routing_integration branch from 3fbdd04 to ef83df2 Compare August 11, 2026 19:10
@kohlivrinda
kohlivrinda force-pushed the 08-11-_do-not-merge-yet_complexity_cache-aware_session_routing_integration branch from ef83df2 to 7a3d7cc Compare August 12, 2026 11:31
@kohlivrinda
kohlivrinda force-pushed the 08-11-_do-not-merge-yet_complexity_cache-aware_session_tier_policy branch from 62bf1ca to 10baa8b Compare August 12, 2026 11:31
@Madhuvod
Madhuvod force-pushed the 08-11-_do-not-merge-yet_complexity_cache-aware_session_tier_policy branch from 10baa8b to 78992e7 Compare August 12, 2026 15:14
@Madhuvod
Madhuvod force-pushed the 08-11-_do-not-merge-yet_complexity_cache-aware_session_routing_integration branch from 7a3d7cc to 06cc703 Compare August 12, 2026 15:14
@Madhuvod
Madhuvod force-pushed the 08-11-_do-not-merge-yet_complexity_cache-aware_session_tier_policy branch from 78992e7 to f2dafca Compare August 12, 2026 19:25
@Madhuvod
Madhuvod force-pushed the 08-11-_do-not-merge-yet_complexity_cache-aware_session_routing_integration branch from 06cc703 to 06f15e6 Compare August 12, 2026 19:25
@Madhuvod
Madhuvod force-pushed the 08-11-_do-not-merge-yet_complexity_cache-aware_session_tier_policy branch from f2dafca to b98813a Compare August 12, 2026 22:06
@Madhuvod
Madhuvod force-pushed the 08-11-_do-not-merge-yet_complexity_cache-aware_session_routing_integration branch from 06f15e6 to b52c3cc Compare August 12, 2026 22:06
@Madhuvod
Madhuvod force-pushed the 08-11-_do-not-merge-yet_complexity_cache-aware_session_routing_integration branch from b52c3cc to 97a29d7 Compare August 12, 2026 22:20
@Madhuvod
Madhuvod force-pushed the 08-11-_do-not-merge-yet_complexity_cache-aware_session_tier_policy branch from b98813a to 5f49487 Compare August 12, 2026 22:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants