Repository navigation
feat(loop): derive the prompt context budget from the model's advertised window #8053
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
henrypark133
wants to merge
29
commits into
main
Choose a base branch
from
context-length
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
29 commits
Select commit
Hold shift + click to select a range
4bede65
feat(loop-contracts): derive a prompt context budget from an advertis…
henrypark133 198974f
feat(loop-contracts): carry a resolved context budget on LoopRunContext
henrypark133 328ffaa
refactor(agent-loop): pass the compaction budget as an argument
henrypark133 267b0b5
feat(agent-loop): honor a run-resolved context budget when compacting
henrypark133 c98383f
feat(loop-host): expose the provider-advertised context window on the…
henrypark133 2100238
refactor(loop-host): thread a prompt context budget to the model port
henrypark133 b963204
feat(turn-runner): resolve the prompt context budget from the run's m…
henrypark133 1c714d2
test(support): let the scripted model advertise a context window
henrypark133 2888785
test(loop-host): pin that the gateway wrapper applies its prompt cont…
henrypark133 55ff1c9
test(turn-runner): pin that the derived budget sizes the host's outbo…
henrypark133 d293fe1
test(integration): prove the model-derived context budget through a r…
henrypark133 adfc802
docs: record the context_length consumer and tidy review leftovers
henrypark133 c842ed0
refactor(loop-contracts): move context_budget tests out of line
henrypark133 b687623
chore(architecture-tests): re-pin ironclaw_loop_contracts ceiling at …
henrypark133 ca68ac4
docs(internal): design note and plan for the model-derived context bu…
henrypark133 120402e
fix(llm): keep model_metadata() free of token refresh I/O
henrypark133 ec9ead5
refactor(agent-loop): move compaction strategy tests out of line
henrypark133 3ad5c72
ci(reborn-plan): register tests/trace_llm_tests.rs as a root test par…
henrypark133 2ced021
fix(loop-contracts): treat a window too small for any transcript as u…
henrypark133 2b35d1e
test(integration): read the last captured model request, not index 5
henrypark133 b1f172f
fix(llm): never advertise a guessed or too-large context window
henrypark133 0eb4701
fix(agent-loop): bound the compaction preserve tail by the run's visi…
henrypark133 03bf75c
Merge origin/main into context-length
henrypark133 a2699f9
test(webui): use the live extension id in the notification-setup boun…
henrypark133 cd277c0
fix(llm): FailoverProvider advertises the smallest window across its …
henrypark133 c7af1f1
test(loop-host): pin the window probe never refreshes a token, throug…
henrypark133 1f870b7
refactor(llm): query each failover member once when advertising the w…
henrypark133 f0cdace
ci(nextest): give the whole-tree architecture scans real timeout head…
henrypark133 16c51cc
test(loop-host): pin the window probe's snapshot-identity and metadat…
henrypark133 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
112 changes: 112 additions & 0 deletions
112
crates/contracts/ironclaw_loop_contracts/src/context_budget/tests.rs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,112 @@ | ||
| use super::*; | ||
|
|
||
| #[test] | ||
| fn visible_transcript_tokens_reserves_larger_output_buffer() { | ||
| let budget = PromptContextTokenBudget::new(100, 10, 30); | ||
|
|
||
| assert_eq!(budget.visible_transcript_tokens(), 70); | ||
| } | ||
|
|
||
| #[test] | ||
| fn visible_transcript_tokens_saturates_when_reserve_exceeds_limit() { | ||
| let budget = PromptContextTokenBudget::new(10, 20, 0); | ||
|
|
||
| assert_eq!(budget.visible_transcript_tokens(), 0); | ||
| } | ||
|
|
||
| #[test] | ||
| fn visible_transcript_tokens_uses_reserve_when_larger_than_output_budget() { | ||
| let budget = PromptContextTokenBudget::new(100, 30, 10); | ||
|
|
||
| assert_eq!(budget.visible_transcript_tokens(), 70); | ||
| } | ||
|
|
||
| #[test] | ||
| fn advertised_window_of_none_reproduces_the_compiled_in_default() { | ||
| // A provider that reports nothing must behave exactly as it does | ||
| // today. This is the compatibility guarantee of the whole change. | ||
| assert_eq!( | ||
| PromptContextTokenBudget::from_advertised_window(None), | ||
| PromptContextTokenBudget::default() | ||
| ); | ||
| } | ||
|
|
||
| #[test] | ||
| fn advertised_window_of_zero_is_treated_as_unknown() { | ||
| assert_eq!( | ||
| PromptContextTokenBudget::from_advertised_window(Some(0)), | ||
| PromptContextTokenBudget::default() | ||
| ); | ||
| } | ||
|
|
||
| #[test] | ||
| fn large_advertised_window_keeps_the_flat_response_reserve() { | ||
| let budget = PromptContextTokenBudget::from_advertised_window(Some(2_000_000)); | ||
|
|
||
| assert_eq!(budget.context_limit_tokens, 1_800_000); | ||
| assert_eq!( | ||
| budget.reserve_tokens, | ||
| PromptContextTokenBudget::DEFAULT_RESERVE_TOKENS | ||
| ); | ||
| assert_eq!(budget.visible_transcript_tokens(), 1_780_000); | ||
| } | ||
|
|
||
| #[test] | ||
| fn small_advertised_window_clamps_the_reserve_and_keeps_budget_usable() { | ||
| // An 8k model would otherwise have its entire budget consumed by the | ||
| // flat 20k response reserve, leaving zero visible transcript and a | ||
| // loop that cannot run at all. | ||
| let budget = PromptContextTokenBudget::from_advertised_window(Some(8_000)); | ||
|
|
||
| assert_eq!(budget.context_limit_tokens, 7_200); | ||
| assert_eq!(budget.reserve_tokens, 1_800); | ||
| assert!( | ||
| budget.visible_transcript_tokens() > 0, | ||
| "a small-window model must still have room for transcript" | ||
| ); | ||
| } | ||
|
|
||
| #[test] | ||
| fn smallest_positive_window_is_treated_as_unknown() { | ||
| // Some(1) survives the `> 0` filter but derives a zero visible | ||
| // transcript, which must fall back to the default exactly like | ||
| // None and Some(0). | ||
| assert_eq!( | ||
| PromptContextTokenBudget::from_advertised_window(Some(1)), | ||
| PromptContextTokenBudget::default() | ||
| ); | ||
| } | ||
|
|
||
| #[test] | ||
| fn smallest_usable_window_keeps_a_nonzero_visible_transcript() { | ||
| // Find the smallest advertised window whose derivation does NOT | ||
| // fall back to the default, and prove it still leaves visible | ||
| // transcript room rather than trusting the arithmetic. | ||
| let smallest_non_default = (1..=16) | ||
| .find(|&candidate| { | ||
| PromptContextTokenBudget::from_advertised_window(Some(candidate)) | ||
| != PromptContextTokenBudget::default() | ||
| }) | ||
| .expect("some small window must derive a non-default budget"); | ||
|
|
||
| let budget = PromptContextTokenBudget::from_advertised_window(Some(smallest_non_default)); | ||
|
|
||
| assert_ne!(budget, PromptContextTokenBudget::default()); | ||
| assert!( | ||
| budget.visible_transcript_tokens() > 0, | ||
| "the smallest non-default derived budget must still leave visible transcript room" | ||
| ); | ||
| } | ||
|
|
||
| #[test] | ||
| fn advertised_window_matching_todays_constant_is_reduced_by_the_margin() { | ||
| // 128k advertised is NOT the same as the 128k fallback: the fallback | ||
| // is a guess, an advertised value gets the estimate-error margin. | ||
| let budget = PromptContextTokenBudget::from_advertised_window(Some(128_000)); | ||
|
|
||
| assert_eq!(budget.context_limit_tokens, 115_200); | ||
| assert_eq!( | ||
| budget.reserve_tokens, | ||
| PromptContextTokenBudget::DEFAULT_RESERVE_TOKENS | ||
| ); | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.