Skip to content

feat: log compression attempt telemetry - #60444

Closed
deltaahead wants to merge 1 commit into
NousResearch:mainfrom
deltaahead:obs/compression-telemetry-v2
Closed

feat: log compression attempt telemetry#60444
deltaahead wants to merge 1 commit into
NousResearch:mainfrom
deltaahead:obs/compression-telemetry-v2

Conversation

@deltaahead

@deltaahead deltaahead commented Jul 7, 2026

Copy link
Copy Markdown

Summary

  • Adds one content-free JSON log line for each Hermes compression attempt with attempt/session IDs, trigger source, token estimates, aux model metadata, durations, fallback flag, and split/commit status.
  • Records protected head/tail and middle-window token estimates without logging prompt text, summaries, or transcript content.
  • Covers metadata-only telemetry and aux-call duration capture with synthetic compression tests.

Scope

  • Observability/logging only.
  • No compression behavior, model/provider, threshold/autoraise, fallback, chunking, config, .env, or project repo changes.

Test Plan

  • ........................................................................ [ 44%]
    ........................................................................ [ 89%]
    ................. [100%]
    161 passed in 9.66s
  • All checks passed!
  • Synthetic telemetry sample confirmed no transcript or summary content.

@deltaahead deltaahead mentioned this pull request Jul 7, 2026
Closed
@alt-glitch alt-glitch added type/feature New feature or request comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state P2 Medium — degraded but workaround exists labels Jul 7, 2026
teknium1
teknium1 reviewed Jul 15, 2026
View reviewed changes

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the focused, content-free telemetry work. One coverage gap needs resolving before this can substantiate its “each Hermes compression attempt” claim.

Problems

  • agent/conversation_compression.py:504 initializes telemetry, but the Codex app-server branch at agent/conversation_compression.py:525 immediately returns through _compress_context_via_codex_app_server. Current main performs native compaction at agent/conversation_compression.py:1173 and records a successful boundary at agent/conversation_compression.py:1199; neither the success nor failure path reaches the proposed emitter.
  • tests/agent/test_compression_attempt_telemetry.py exercises only the ordinary compressor path, so this omission is untested.

Suggested changes

  • Cover Codex-native success, interruption/error, and skipped outcomes with one metadata-only record per attempt, and add focused regression tests.

Automated hermes-sweeper review.

Comment thread agent/conversation_compression.py
@@ -465,6 +504,19 @@ def compress_context(
prompt — the session is NOT rotated. Callers should detect the

@teknium1 teknium1 Jul 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This telemetry setup is bypassed by the existing Codex app-server return at line 525. Native compaction therefore produces no record even though it is a real compression path; emit telemetry from _compress_context_via_codex_app_server for its success and failure outcomes too.

@teknium1 teknium1 added sweeper:blast-broad Sweeper blast radius: broad — a core path most sessions hit area/compression Context compression and continuation sessions labels Jul 15, 2026
@teknium1 teknium1 mentioned this pull request Jul 22, 2026
Merged
teknium1 added a commit that referenced this pull request Jul 22, 2026
@teknium1
fix: adapt compression attempt logging to current main aux-call contract
aa07207 
- aux summary call on main intentionally omits max_tokens; use .get() in the
  telemetry hook (and widen the param type) so the hook never breaks the call
- update test expectation: aux_output_reservation is None on main
- record no_progress failure_class in the no-progress boundary branch

Follow-up for salvaged PR #60444.
teknium1 added a commit that referenced this pull request Jul 22, 2026
@teknium1
fix: adapt compression attempt logging to current main aux-call contract
cbc1054 
- aux summary call on main intentionally omits max_tokens; use .get() in the
  telemetry hook (and widen the param type) so the hook never breaks the call
- update test expectation: aux_output_reservation is None on main
- record no_progress failure_class in the no-progress boundary branch

Follow-up for salvaged PR #60444.
@teknium1

teknium1 commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Merged via #69338 (commit cbc1054). Your commit was cherry-picked with authorship preserved (verified egress-free and content-free — the parking policy for outbound telemetry didn't apply to local logging). One adaptation: the aux call kwargs access was hardened against main's no-max-tokens summary contract. Thanks!

@teknium1 teknium1 closed this Jul 22, 2026
randlee pushed a commit to randlee/hermes-agent that referenced this pull request Aug 11, 2026
@teknium1
fix: adapt compression attempt logging to current main aux-call contract
90b5dc1 
- aux summary call on main intentionally omits max_tokens; use .get() in the
  telemetry hook (and widen the param type) so the hook never breaks the call
- update test expectation: aux_output_reservation is None on main
- record no_progress failure_class in the no-progress boundary branch

Follow-up for salvaged PR NousResearch#60444.
33hodl pushed a commit to 33hodl/hermes-agent that referenced this pull request Aug 12, 2026
@teknium1
fix: adapt compression attempt logging to current main aux-call contract
dabb789 
- aux summary call on main intentionally omits max_tokens; use .get() in the
  telemetry hook (and widen the param type) so the hook never breaks the call
- update test expectation: aux_output_reservation is None on main
- record no_progress failure_class in the no-progress boundary branch

Follow-up for salvaged PR NousResearch#60444.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@teknium1 teknium1 teknium1 left review comments

Assignees

No one assigned

Labels

area/compression Context compression and continuation sessions comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists sweeper:blast-broad Sweeper blast radius: broad — a core path most sessions hit sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/feature New feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@deltaahead @teknium1 @alt-glitch