Skip to content

fix(compaction): gate in-turn stats on announce render-eligibility + self-identifying markers - #177

Merged
Kyzcreig merged 2 commits into
mainfrom
fix/inturn-stats-render-gate
Jul 2, 2026
Merged

Kyzcreig merged 2 commits into
mainfrom
fix/inturn-stats-render-gate

Conversation

@Kyzcreig

@Kyzcreig Kyzcreig commented Jul 2, 2026

Copy link
Copy Markdown
Collaborator

Problem

The in-turn compaction-stats block runs for EVERY LCM compress() call — including no-ops — while the announce formatter default-denies non-rendering statuses. Result: COMPACTION_STATS_APPROX_ATTRIBUTION (at ~100% kept_tail) and TAG_MISSING WARNINGs fire for announces that never render. Measured 2026-07-02: every real-session degrade marker in apollo's agent.log rode an LCM compression no-op line — pure log noise polluting the daily compaction-stats watcher report.

Fix

  • P1 render gate: _inturn_stats_render_eligible() mirrors the formatter's own gating by consuming the SAME allow-list objects + the conditional post<pre token check, applied to the exact variables the announce call passes as pre_tokens/post_tokens. LCM-scoped (built-in path unchanged — it keeps sid-rotation gating and may not expose _last_compression_status).
  • D-4 marker self-identification: every compaction-stats marker now carries session=<id> + src=test under pytest, so log consumers attribute markers by the line's own fields (pytest writing through the live logging config previously inflated production counts).

Tests

13 new: gate matrix vs the real formatter as oracle; drift guards (gate uses the formatter's allow-list objects + the announce-call token variables — source-structure); stats=None render tolerance; marker fields incl. from a threading.Thread worker (the measured polluter's shape); throttle unaffected.

Spec: plans/2026-07-02_inturn-noop-gate-multipass-provenance-SPEC.md v0.5 — 4 Opus review passes (BLOCK → AWC → BLOCK → AWC/zero-blockers). PR-A of the pair; PR-B (multi-pass provenance shadow) stacks on this.

…self-identifying markers

The in-turn compaction-stats block (build_inturn_stats + gross-error guard +
COMPACTION_STATS_* WARNINGs) ran for EVERY LCM compress() call — including
no-ops — while the announce formatter default-denies non-rendering statuses
(noop/idle/running/bypassed; conditional statuses failing the post<pre render
check). Result: ~100% kept_tail APPROX_ATTRIBUTION and TAG_MISSING WARNINGs
fired for announces that never render — pure log noise that polluted the
daily compaction-stats watcher report (measured 2026-07-02: every real-session
degrade marker in apollo's agent.log rode an 'LCM compression no-op' line).

Fix: _inturn_stats_render_eligible() mirrors the formatter's own gating by
consuming the SAME allow-list objects (single source of truth, no copied
literals) + the conditional token check, applied on the exact variables the
announce call passes as pre_tokens/post_tokens. LCM-scoped: the built-in path
keeps its sid-rotation gating and may not expose _last_compression_status.

Also (D-4): every compaction-stats marker now carries session=<id>, plus
src=test under pytest (PYTEST_CURRENT_TEST), so the log watcher can attribute
markers by the line's own fields instead of fragile proximity joins — pytest
writing through the live logging config previously inflated production counts.

Tests: 13 new (gate predicate matrix vs the real formatter as oracle; drift
guards asserting the gate uses the formatter's allow-list objects and the
announce-call token variables; stats=None render tolerance; marker session/
src=test fields incl. from a threading.Thread worker — the measured polluter's
shape; throttle unaffected). Spec (4 Opus review passes):
plans/2026-07-02_inturn-noop-gate-multipass-provenance-SPEC.md v0.5, PR-A.
@greptile-apps

greptile-apps Bot commented Jul 2, 2026 •

Copy link
Copy Markdown

Greptile Summary

This PR introduces a render-eligibility gate (_inturn_stats_render_eligible) that stops the in-turn compaction-stats build from running on LCM no-op/idle/running/bypassed compactions that the formatter would never display, eliminating spurious COMPACTION_STATS_APPROX_ATTRIBUTION and TAG_MISSING warnings. It also appends session=<id> and src=test self-identification fields to every _warn_compaction_stats_once marker so log consumers can attribute and exclude test noise without fragile proximity joins.

  • Render gate (_inturn_stats_render_eligible): mirrors _format_compaction_announce's LCM allow-list gating exactly; the non-LCM (built-in compressor) path is left unconditionally eligible to preserve existing behavior.
  • Marker self-identification: suffix is appended after the throttle key computation (keyed on the original message's first two tokens), so the throttle continues to deduplicate correctly while every emitted log line now carries the session id and, under pytest, a src=test flag.

Confidence Score: 5/5

Safe to merge. The gate is a pure noise-reduction change on the LCM path, the non-LCM path is explicitly left unchanged, and gate correctness is verified against the real formatter as oracle.

The gate predicate wraps the same allow-list objects the formatter already uses, so gate and formatter can only disagree if the module-level sets change — enforced by the drift-guard test. The suffix appended to warning markers is isolated to a best-effort try/except after the throttle key computation, so it cannot affect deduplication or break the reply path. No pre-existing code path is altered for the non-LCM compressor.

No files require special attention for merge safety. The regex in test_stats_gate_token_identity_and_lcm_scope is worth hardening before the next formatter pass but does not block this PR.

Important Files Changed

Filename Overview
agent/conversation_compression.py Adds _inturn_stats_render_eligible predicate and LCM-scoped gate in compress_context; appends session/src suffix to _warn_compaction_stats_once. Non-LCM path left unchanged (always eligible).
tests/agent/test_inturn_stats_render_gate.py 13 new tests covering gate predicate, formatter-oracle agreement, source-structure drift guards, marker self-id fields, and throttle correctness. test_stats_gate_token_identity_and_lcm_scope uses a regex anchored to production source whitespace, making it sensitive to formatting changes.

Reviews (2): Last reviewed commit: "fix(compaction): don't suppress non-LCM ..." | Re-trigger Greptile

Comment thread agent/conversation_compression.py Outdated
Comment thread tests/agent/test_inturn_stats_render_gate.py Outdated
…ptile #177)

The render gate was written as (_engine_name == 'lcm') and render_eligible(...),
which short-circuits to False for the built-in (non-LCM) compressor and
permanently dropped its stats — silently degrading every built-in announce to
the two-line form. Intent was to gate LCM no-ops only. Restructure to an
explicit branch: LCM → render-eligibility; non-LCM → always eligible (unchanged
always-attempt behavior).

Also closes the Greptile test-coverage gap: _run_announce_block now mirrors the
real LCM-scoped gate and takes explicit pre/post tokens, and new tests exercise
(a) a conditional status ('sanitized') with a REAL compression post<pre through
build+validate, (b) a conditional no-op not built, and (c) non-LCM not
suppressed by the gate.
@Kyzcreig

Kyzcreig commented Jul 2, 2026

Copy link
Copy Markdown
Collaborator Author

Both Greptile P2 findings addressed in d303095:

  1. Non-LCM stats suppression (real regression — good catch): the gate was (_engine_name == "lcm") and render_eligible(...), which short-circuited to False for the built-in compressor and dropped its stats entirely. Restructured to an explicit branch — LCM gets render-eligibility, non-LCM stays True (unchanged always-attempt). The source-structure contract test now asserts this shape incl. the non-LCM else: eligible = True.
  2. Conditional-status coverage gap: _run_announce_block now mirrors the real LCM-scoped gate and takes explicit pre/post tokens; added test_inturn_stats_built_on_conditional_real_compression (sanitized status, real post<pre, through build+validate), a conditional-noop-not-built case, and test_nonlcm_stats_not_suppressed_by_gate.

@Kyzcreig
Kyzcreig merged commit ff799fb into main Jul 2, 2026
35 checks passed
@Kyzcreig
Kyzcreig deleted the fix/inturn-stats-render-gate branch July 2, 2026 21:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant