Skip to content

fix(engine): charge the carried conversation from the map that holds it - #2278

Merged
justinchuby merged 1 commit into
mainfrom
squad/gaff-carry-gate-under-reserve
Aug 27, 2026
Merged

justinchuby merged 1 commit into
mainfrom
squad/gaff-carry-gate-under-reserve

Conversation

@justinchuby

Copy link
Copy Markdown
Owner

Closes #2251.

The defect

admit_interpreted_generate_request gated the carried prefix length on one map and read its value from another:

let carried = if self.workflow_sessions.contains_key(&session_id) {   // engine-side, SessionId
    self.workflow.session_prepended_prompt_len(&session_id.to_string())  // pipeline-side, (String, cell)
} else {
    0
};

session_prepended_prompt_len is already total — let Some(cell) = … else { return 0 } and .unwrap_or(0), so it answers 0 for a session it does not know. That makes the gate's else arm identical to the callee's own answer:

  • maps agree → the gate is redundant, the callee returns the same value;
  • maps diverge (workflow holds a prefix, engine map does not) → the gate discards a correct non-zero count and forces carried = 0.

No input exists for which the gate improves the outcome. Its only reachable effect is to under-reserve — exactly what the function's own comment warns about:

Admitting on the request alone under-reserves for exactly the turns that need the most.

The fix

Read it unconditionally. The sole caller is the workflow path, so no non-workflow hot path pays for the removal. Production change is 6 lines out, 3 lines in.

Why this needed a new test rather than an existing one

This is latent, not live — close_session maintains both sides, and the caller-side hazard is already closed. So the agreeing case cannot distinguish the two versions, which is precisely why deleting the gate leaves the rest of the suite green. A change that no test can fail is a change with no evidence.

The new test builds the divergence directly: create a session, seed its conversation, then drop it from the engine-side map only — the state any future close/reset/eviction path that skipped forget_session would leave behind.

Confirmed failing before the fix, on the same commit:

a forgotten session holding 3 tokens: the carried conversation was not charged,
so a turn over budget was admitted; got: <admitted>
test result: FAILED. 0 passed; 1 failed; 668 filtered out

A turn needing 5 tokens of budget was admitted against a budget of 3. The failure mode is silent by nature — an admission that should have been a refusal, not an error.

Two controls make the arms mean something, both asserted rather than assumed:

  • workflow_sessions.remove(...).is_some() — create_session is what inserted the id, so the removal is real. A silently-failed removal would send both arms down the agreeing path and pass while proving nothing.
  • session_prepended_prompt_len(...) == seed.len() — the conversation is still readable from the map the carry is read from, per arm.

Each arm gets its own engine. admit_* takes a reservation that only complete() returns, so a shared engine would let the admitted arm's leftover reservation pay for the other arm's refusal — and the test would pass with the carry ignored.

Verification

check result
new test, before fix FAILED (the falsifier above)
new test, after fix ok — 1 passed; 668 filtered out
a_continuing_turn_is_admitted_for_the_conversation_it_carries (#1982, the agreeing case) ok — behaviour-identical today
full engine lib suite 668 passed; 0 failed; 1 ignored
cargo fmt --check clean
cargo clippy --all-targets -D warnings clean
cargo check at default features (#1973 lane) clean

Both targeted runs used --exact and report 1 passed / 668 filtered out, so the filter demonstrably matched — per the empty-filter hazard scripts/test_step.sh guards against.

Limitations

The divergence is constructed by the test, not reachable through any current public path. This does not fix a live bug; it removes a gate whose correctness depended on an invariant maintained in a different module from the one reading it, with nothing asserting that invariant — and it adds the assertion.

`admit_interpreted_generate_request` gated the carried prefix length on
`workflow_sessions` (engine-side, keyed by `SessionId`) while reading its
value from `workflow.worker.session_state` (pipeline-side, keyed by
`(String, cell)`).

`session_prepended_prompt_len` is already total: it returns 0 for a
session it does not know. So when the two maps agree the gate is
redundant -- the callee returns the same 0 the `else` arm supplies -- and
when they diverge the gate discards a correct non-zero count and forces
`carried = 0`. There is no input for which it improves the outcome; its
only reachable effect is to under-reserve, which is precisely what the
function's own comment warns against.

Read it unconditionally instead. The only caller is the workflow path, so
no non-workflow hot path pays for the removal.

This was latent, not live: `close_session` maintains both sides, and the
caller-side hazard is already closed. But the gate's correctness depended
on an invariant maintained in a different module from the one reading it,
and nothing asserted that invariant -- so a future close/reset/eviction
path that skipped `forget_session` would have silently under-reserved the
KV budget on the longest conversations, failing as an admission that
should have been a refusal rather than as an error.

The agreeing case cannot test a redundant gate, which is why removing it
left the suite green. The new test builds the divergence directly and
fails without this change: a turn needing 5 tokens of budget was admitted
against a budget of 3. Each arm uses its own engine, since `admit_*`
takes a reservation that only `complete()` returns and a shared engine
would let the admitted arm pay for the other arm's refusal.

Closes #2251

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@justinchuby
justinchuby enabled auto-merge (squash) August 27, 2026 10:05
@codecov

codecov Bot commented Aug 27, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.10%. Comparing base (8395917) to head (3ec856f).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #2278      +/-   ##
==========================================
+ Coverage   80.75%   81.10%   +0.34%     
==========================================
  Files         434      434              
  Lines      222170   222170              
  Branches   222170   222170              
==========================================
+ Hits       179416   180188     +772     
+ Misses      36824    36050     -774     
- Partials     5930     5932       +2     
Flag Coverage Δ
cli-ort-linux 72.51% <ø> (ø)
cli-ort-windows 72.10% <ø> (+0.09%) ⬆️
mlas 86.02% <ø> (+0.12%) ⬆️
offline 81.23% <ø> (+0.35%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 8 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@justinchuby
justinchuby merged commit 47a35b8 into main Aug 27, 2026
30 of 31 checks passed
@justinchuby
justinchuby deleted the squad/gaff-carry-gate-under-reserve branch August 27, 2026 11:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

admit_interpreted_generate_request: carry gate reads one session map and is gated on another, and can only under-reserve

1 participant