refactor(context_compressor): extract skill-prune/ghost-skill cluster (LB2) - #80628
Open
andrexibiza wants to merge 2 commits into
Open
refactor(context_compressor): extract skill-prune/ghost-skill cluster (LB2)#80628andrexibiza wants to merge 2 commits into
andrexibiza wants to merge 2 commits into
Conversation
…ll-prune Wave-2 dual REQUEST_CHANGES agreed LB2 skill-prune cannot extract first: it calls _redact_compaction_text and _content_text_for_contains (cycle). Move those two pure leaves into context_compressor_text_utils.py with is-identical re-export through the godfile. Golden bodies byte-exact. Part of NousResearch#78645 Part of NousResearch#78647 Signed-off-by: Andrex Ibiza <andrexibiza@users.noreply.github.com>
… (LB2) Wave-2 redraw: banner-inclusive window 404-618 (golden ec5defbef6a402ad302b28369c81f5f9182162f1c8353b231d2d863fe59305c0) into context_compressor_skill_prune.py. Deps on the two text-util leaves now import one-way from context_compressor_text_utils — no cycle. 12 members re-exported is-identical through the godfile. 35/35 targeted tests green. Part of NousResearch#78645 Part of NousResearch#78647 Signed-off-by: Andrex Ibiza <andrexibiza@users.noreply.github.com>
Contributor
Author
Contributor
Author
|
Interlock completion for the context-compressor shard campaign:
The full related set is explicitly recorded in #81074, including the prior compressor shards, same-surface colliders/fixers, and their referenced issue nodes. The residual |
Contributor
refactor(context_compressor): extract skill-prune/ghost-skill cluster (LB2) Declared god-file extraction with clean module boundaries and
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
God-file Feature Package (tracker #78647):
agent/context_compressor.pyslice LB2 — extract the skill-prune / ghost-skill defense cluster intoagent/context_compressor_skill_prune.pywith anis-identical re-export seam.Slice
agent/context_compressor.py(6,883 lines @ pin6e9cae6ac4b)2b7a1b97807)ec5defbef6a402ad302b28369c81f5f9182162f1c8353b231d2d863fe59305c0— byte-exact, verified by 5/5 blind reviewersagent/context_compressor_skill_prune.py(12 members: 6 constants + 6 functions)context_compressor_text_utils(leaf extract refactor(context_compressor): extract text-util leaves into context_compressor_text_utils #80626) — no cycle (W2 C1 resolved)cc.<m> is sp.<m>verified at runtimeVerification
git diff --checkclean · LF-only · DCO signedPart of #78645
Part of #78647
Prior Work / Attribution
Related work searched (multiple methods, all states, before finalizing this PR):
context_compressor+skill-prune/ghost-skill):agent/context_compressor.py: refactor(context_compressor): extract text-util leaves into context_compressor_text_utils #80626 (LB text-utils), refactor(context_compressor): extract skill-prune/ghost-skill cluster (LB2) #80628 (LB2, this), refactor(context_compressor): extract token-budget estimators into context_compressor_budget (LB3) #80634 (LB3 budget), refactor(context_compressor): extract content/media strip helpers (LB4) #80636 (LB4 strip), refactor(context_compressor): extract threshold coercion mixin (LB7) #80644 (LB7 threshold), refactor(context_compressor): extract tool-result summarizers (LB5a) #80645 (LB5a), refactor(context_compressor): extract durable session guards mixin (LB6) #81074 (LB6 guards), refactor(context_compressor): extract summary kernel and preserve compatibility #81181 (summary kernel), refactor(context_compressor): extract message marker helpers #81243 (message markers), plus shard tracker Shard agent/context_compressor.py (god-file decomposition) #78645 and epic [COMPLETE] Large-file decomposition: 20/20 done #78647.agent/context_compressor.pyforSKILL_PRUNED/ghost-skill: confirmed the extracted cluster is the merged ghost-skill defense implementation (markers, protected prune, deterministic survival).agent/context_compressor_skill_prune.pyand its seam test are touched by no other open PR (collision-clean). The godfile is shared across the shard series — expected; each PR extracts a disjoint cluster.Unlinked related work (stars not bound to this PR by keywords):
fix(compression): ghost-skill defense — [SKILL_PRUNED] markers, protected prune, deterministic survival— the merged origin of the cluster this PR verbatim-extracts. Credited; no duplicate filed; no code copied — pure extraction of already-shipped work.Builds on: #80626 (
context_compressor_text_utilsleaf dependency, one-way, no cycle).Merge-order / dependency: merge #80626 before #80628.
Duplicates: none — no existing PR covers the LB2 skill-prune/ghost-skill cluster.
Prior Work / Attribution
Prior credit — the humans whose work this builds on, before anyone else:
If a related artifact has no attributable human author, say so plainly rather than crediting a ticket number as if it were a person.
Related work searched (multiple methods, all states, before finalizing this PR):
context_compressor+skill-prune/ghost-skill): 16 open extraction shards onagent/context_compressor.py(LB2 this, refactor(context_compressor): extract text-util leaves into context_compressor_text_utils #80626 text-utils, refactor(context_compressor): extract token-budget estimators into context_compressor_budget (LB3) #80634 budget, refactor(context_compressor): extract content/media strip helpers (LB4) #80636 strip, refactor(context_compressor): extract threshold coercion mixin (LB7) #80644 threshold, refactor(context_compressor): extract tool-result summarizers (LB5a) #80645 summarizers, refactor(context_compressor): extract durable session guards mixin (LB6) #81074 guards, refactor(context_compressor): extract summary kernel and preserve compatibility #81181 summary kernel, refactor(context_compressor): extract message marker helpers #81243 message markers, plus conversation_loop siblings refactor(conversation_loop): extract streaming spinner cleanup slice CL-R2-1 into agent/streaming_control.py #84310/refactor(conversation_loop): extract success cleanup slice CL-R3-2 into agent/conversation_loop_success_cleanup.py #84653/refactor(conversation_loop): extract error classifier slice CL-R1-2 into agent/conversation_error_classifiers.py #84619/refactor(conversation_loop): extract turn persistence slice CL-R5-1 into agent/turn_finalizer.py #84583/refactor(conversation_loop): extract text helper slice CL-R1-1 into agent/conversation_text.py #84275); bug landscape fix: add usedforsecurity=False to hashlib.md5 in context_compressor (#FIPS) #74632/fix: hashlib.md5 FIPS crash in context_compressor.py #56715 (md5 FIPS), feat(context_compressor): add compression efficiency ratio logging #29859 (ratio logging), fix(context_compressor): harden tool-call previews in summaries #12588, fix(aux): handle dict/str message shape in extract_content_or_reasoning #55941, Compression trigger prices every image at a flat 1500 tokens — vision-heavy sessions on a 64K local model hit provider 400s before compaction can fire #70328, [Bug] Context compressor ignores reasoning field — empty summaries with thinking models (Ollama 0.22+) #19003, Compression abort can reuse stale in-place compaction state #58630, Context compression silently drops all conversation turns when summary generation fails #10719, [Bug] StepFun API: Unterminated string error caused by broken JSON in context_compressor #12242, fix(cli): reinitialize context compressor on /new to pick up updated context length #31067, fix(context_compressor): thread custom_providers to get_model_context_length #49193, feat(agent): make MINIMUM_CONTEXT_LENGTH configurable via HERMES_MIN_CONTEXT_LENGTH #44932, context_compressor: preserve absolute threshold override across model switches #17883, fix: redact status --all API keys, harden compression message handling (salvage #15795) #55576, Fix long-context compression with hierarchical summaries #25716, fix(auxiliary): cap concurrent async_call_llm to prevent provider 429 cascade #60311, feat(compression): preserve fidelity across repeated compactions #62777; tracker Shard agent/context_compressor.py (god-file decomposition) #78645; tracker [COMPLETE] Large-file decomposition: 20/20 done #78647.agent/context_compressor.pyforSKILL_PRUNED/ghost-skill/_skill_pruned_marker: confirmed the extracted cluster is the merged ghost-skill defense implementation from @teknium1's fix(compression): ghost-skill defense — [SKILL_PRUNED] markers, protected prune, deterministic survival (salvage #44166) #70275 (markers, protected prune, deterministic survival).agent/context_compressor_skill_prune.pyand its seam test touched by no other open PR (collision-clean). The godfile is shared across the shard series — expected; each PR extracts a disjoint cluster.Unlinked related work (stars not bound to this PR by keywords):
Builds on: @andrexibiza, #80626 (
context_compressor_text_utilsleaf dependency, one-way, no cycle).Merge-order / dependency: merge #80626 before #80628.
Duplicates: none — no existing PR covers the LB2 skill-prune/ghost-skill cluster.