feat(retrieval): pack-loop budget uses compressed cost when flag ON (#434) - #497
Merged
Merged
Conversation
Phase 2 of #434. retrieve_with_tiers now accepts use_type_aware_compression and accounts for L2.5/L1/BFS beliefs at their compress_for_retrieval rendered_tokens during pack accounting, not raw _belief_tokens(b). Locks always render verbatim per the strategy table, so locked accounting is unchanged. retrieve_v2 threads its existing flag through to retrieve_with_tiers instead of compressing only post-pack. Result: at fixed budget, the pack admits strictly more beliefs when ON, closing the A2 wider-pack precondition cited in feature-type-aware-compression.md. Default-OFF preserves byte-identical selection — covered by the test_pack_byte_identical_when_flag_off invariant. The test_pack_widens_when_flag_on test fires at budget=80 against a mixed fact+transient corpus where ON admits 5 stub-cost transients that OFF cannot fit at raw cost.
Phase 1 (compress_for_retrieval + flag wiring) shipped at a6f1582; phase 2 (pack-loop budget consumes rendered_tokens) lands here. Status header reflects: implementation done, default-OFF flag held, lab bench A2 / A4 still pending before flag flip.
Reviewer's GuideImplements phase 2 of type-aware compression for retrieval by making pack-loop budgeting optionally use compressed token costs while keeping behavior byte-identical when the flag is off, adds tests to verify widening behavior and invariants, and updates the feature spec docs to reflect the implementation status. Sequence diagram for retrieval_v2 using type-aware compression in pack-loop budgetingsequenceDiagram
actor Client
participant RetrievalAPI as retrieve_v2
participant RetrievalCore as retrieve_with_tiers
participant Store as BeliefStore
participant Compressor as compress_for_retrieval
Client->>RetrievalAPI: retrieve_v2(query, use_type_aware_compression)
RetrievalAPI->>RetrievalCore: retrieve_with_tiers(..., use_type_aware_compression)
activate RetrievalCore
RetrievalCore->>RetrievalCore: compress_on = resolve_use_type_aware_compression(use_type_aware_compression)
RetrievalCore->>Store: list_locked_beliefs()
Store-->>RetrievalCore: locked_beliefs
loop For each unlocked L25 belief
alt compress_on is False
RetrievalCore->>RetrievalCore: cost = _belief_tokens(b)
else compress_on is True
RetrievalCore->>Compressor: compress_for_retrieval(b, locked=False)
Compressor-->>RetrievalCore: compressed_belief
RetrievalCore->>RetrievalCore: cost = compressed_belief.rendered_tokens
end
RetrievalCore->>RetrievalCore: used += cost
end
loop For each L1 belief
alt compress_on is False
RetrievalCore->>RetrievalCore: cost = _belief_tokens(b)
else compress_on is True
RetrievalCore->>Compressor: compress_for_retrieval(b, locked=False)
Compressor-->>RetrievalCore: compressed_belief
RetrievalCore->>RetrievalCore: cost = compressed_belief.rendered_tokens
end
RetrievalCore->>RetrievalCore: if used + cost <= budget then pack belief
end
loop For each BFS hop belief
alt compress_on is False
RetrievalCore->>RetrievalCore: cost = _belief_tokens(b)
else compress_on is True
RetrievalCore->>Compressor: compress_for_retrieval(b, locked=False)
Compressor-->>RetrievalCore: compressed_belief
RetrievalCore->>RetrievalCore: cost = compressed_belief.rendered_tokens
end
RetrievalCore->>RetrievalCore: if used + cost <= budget then pack belief
end
RetrievalCore-->>RetrievalAPI: packed_beliefs
deactivate RetrievalCore
RetrievalAPI-->>Client: RetrievalResult(packed_beliefs, compressed_beliefs unchanged)
Class diagram for retrieval pack-loop with optional type-aware compressionclassDiagram
class RetrievalModule {
+retrieve_with_tiers(store, query, entity_index_enabled, posterior_weight, use_bm25f_anchors, bm25f_cache, heat_kernel_enabled, eigenbasis_cache, use_type_aware_compression) tuple
+retrieve_v2(store, query, include_locked, entity_index_enabled, posterior_weight, use_bm25f, bm25f_cache, use_type_aware_compression) tuple
-_cost(belief) int
}
class Belief {
+id str
+lock_level int
}
class CompressedBelief {
+rendered_tokens int
}
class BeliefStore {
+list_locked_beliefs() list~Belief~
}
class CompressionModule {
+compress_for_retrieval(belief, locked) CompressedBelief
}
RetrievalModule ..> Belief : uses
RetrievalModule ..> BeliefStore : uses
RetrievalModule ..> CompressionModule : calls
RetrievalModule ..> CompressedBelief : uses
CompressionModule ..> Belief : compresses
BeliefStore o--> Belief : contains
class LockLevels {
<<enumeration>>
LOCK_USER
}
Belief --> LockLevels : uses
class FlagsResolver {
+resolve_use_type_aware_compression(use_type_aware_compression) bool
}
RetrievalModule ..> FlagsResolver : resolves_flag
%% _cost behavior
class CostBehavior {
+_cost(belief) int
-compress_on bool
}
CostBehavior ..> Belief : input
CostBehavior ..> CompressionModule : optional_compression
CostBehavior ..> CompressedBelief : reads_rendered_tokens
RetrievalModule ..> CostBehavior : delegates_cost_calculation
File-Level Changes
Possibly linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
|
Warning Rate limit exceeded
You’ve run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
robotrocketscience
added
author-Kulili
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
Owner
Author
|
[claim:review:Gylf:2026-05-08T19:48:35Z] |
Owner
Author
|
Reviewed at Substance. Closes the A2-precondition for #434 — at fixed budget, ON admits more beliefs than OFF (
Cost-recompute observation (non-blocking). Spec acceptance status (per PR body): A1 lab-side; A2 pack-loop precondition closed here; A3 carries forward; A4 / A5 wait on lab cut + #154. Honest framing — the umbrella stays open and doesn't promise more than landed. Merging. |
Owner
Author
|
[release:review:Gylf:2026-05-08T19:51:32Z] |
This was referenced May 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 2 of #434 — pack-loop budget rewrite. Phase 1 (compress_for_retrieval module + flag wiring + RetrievalResult.compressed_beliefs) shipped at a6f1582. This PR closes the A2 wider-pack precondition: at fixed budget, ON admits more beliefs than OFF.
Change
retrieve_with_tiers()acceptsuse_type_aware_compression: bool | None. When ON, pack accounting for L2.5 / L1 / BFS usescompress_for_retrieval(b, locked=...).rendered_tokensinstead of_belief_tokens(b). Locked beliefs always render verbatim, so locked accounting is unchanged.retrieve_v2()threads its existing flag through. The post-packcompressed_beliefsfield is unchanged.Default-OFF byte-identity
_cost(b)collapses to_belief_tokens(b)whencompress_onresolves False, so the OFF-path arithmetic is identical to pre-#434-phase-2. Covered bytest_pack_byte_identical_when_flag_offand the unmodified existing 2864-test baseline (now 2897 / 43 skipped, full suite green).ON widens
test_pack_widens_when_flag_onatbudget=80against fact + 5×transient: OFF packs 1 belief (fact at ~30 tokens), ON packs ≥ 2 (fact + ≥1 transient at ~10-token stub). Strict inequality.Acceptance against spec
AELFRICE_CORPUS_ROOTlab cutRefs
docs/feature-type-aware-compression.md§ "Where compression sits"Closes A2-precondition for #434. Issue stays open as umbrella for the lab-side bench cut.
Summary by Sourcery
Update retrieval pack-loop budgeting to optionally use type-aware compression while preserving default behavior.
New Features:
Enhancements:
Documentation:
Tests: