fix(cache): scope auxiliary Codex adapter's prompt_cache_key by session - #11
Closed
JoaoMarcos44 wants to merge 1 commit into
Closed
JoaoMarcos44 wants to merge 1 commit into
JoaoMarcos44 wants to merge 1 commit into
Conversation
Fixes NousResearch#79012. compression/flush_memories/MoA/session_search calls went through _CodexCompletionsAdapter, which derived prompt_cache_key from instructions+tools only -- no session scope -- reproducing the NousResearch#78941 bucket-sharing bug on this second code path even after the main transport fix in this PR. set_runtime_main() now threads session_id through (turn_context.py passes agent.session_id); the adapter reads it back via _runtime_main_value("session_id") and scopes the key the same way the main transport does (_cache_scope_from_session_id).
5 tasks
Owner
Author
|
Folded back into the upstream PR (NousResearch#78959) so maintainers review everything in one visible upstream PR instead of fork-only ones. Closing. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes NousResearch#79012. Split out of NousResearch#78959's continuation so it can be reviewed on its own -- see that PR's comment for why this exists.
compression/flush_memories/MoA/session_searchroute through_CodexCompletionsAdapter(agent/auxiliary_client.py), which derivedprompt_cache_keyfrom instructions+tools only -- no session scope at all. Two unrelated sessions whose auxiliary calls share instructions/tools (the common case -- most auxiliary tasks use a small fixed set of system prompts) collapsed onto the same cache bucket, reproducing NousResearch#78941 on this second, less-visible code path.Fix
set_runtime_main()now acceptssession_id(threaded fromagent/turn_context.py's existing call, one line)._CodexCompletionsAdapter.create()reads it back via_runtime_main_value("session_id")and scopes the key through the same_cache_scope_from_session_id()helper the main transport uses.Test plan
tests/agent/test_auxiliary_client.py::TestCodexAuxiliaryAdapterCacheScope— different sessions get different keys; cron re-fires of the same job share one%%{init: {'theme': 'dark', 'themeVariables': { 'primaryColor': '#8b0000', 'mainBkg': '#0a0204', 'primaryTextColor': '#ffccd5', 'primaryBorderColor': '#ff0038', 'lineColor': '#ff0038'}}}%% graph TD A[Session A<br/>compression call] -->|Before fix| X[Shared Cache Bucket<br/>content-only key] B[Session B<br/>flush_memories call] -->|Before fix| X X --> R1[Risk: Cross-Session<br/>Bucket Sharing] A2[Session A] -->|set_runtime_main session_id| Y1[Scoped Key: pck_hash_A] B2[Session B] -->|set_runtime_main session_id| Y2[Scoped Key: pck_hash_B] Y1 --> S1[Isolated Bucket] Y2 --> S2[Isolated Bucket] style X fill:#8b0000,stroke:#ff0038,color:#ffccd5 style R1 fill:#8b0000,stroke:#ff0038,color:#ffccd5 style S1 fill:#3a0000,stroke:#ff0038,color:#ffccd5 style S2 fill:#3a0000,stroke:#ff0038,color:#ffccd5