feat(telemetry): tunable chars-per-token divisor for composition estimate (default 3.5) by Kyzcreig · Pull Request #23 · ANG-Ventures/hermes-agent

Kyzcreig · 2026-06-08T23:38:59Z

What

Makes the chars-per-token divisor used by the request-composition estimate (/context, /usage, /compress) 3.5 by default (was a flat 4), and tunable via env HERMES_COMPOSITION_CHARS_PER_TOKEN (clamped to a sane 2.0–8.0 range; malformed/out-of-range values fall back to the default).

Why

The composition snapshot (PR #21) estimated each bucket at char/4. The classic ~4 chars/token rule is tuned for English prose, but real Hermes requests are dominated by JSON/code-dense tool results that tokenize closer to ~3.5 chars/token — so char/4 systematically under-counts, and the bias grows with the JSON-dense fraction. Measured against live data (this fleet's Discord channel):

measured occupancy	char/4 estimate	delta
253,715	219,266	−13.6%
279,118	236,407	−15.3%
279,650	236,841	−15.3%

Switching to 3.5 tightens the same turns to −1.2% / −3.2% / −3.2% — roughly ±5% across 170k–280k, vs the lopsided −15% before. (Validated 3.5-for-all-buckets against 9 live turns; it beats both flat /4 and a tool-only 3.5 split, and it's one knob instead of two.)

How

agent/model_metadata.py: new module constant COMPOSITION_CHARS_PER_TOKEN (default 3.5), read once at import from HERMES_COMPOSITION_CHARS_PER_TOKEN with a 2.0–8.0 clamp and graceful fallback. compose_request_breakdown's internal _t() now ceiling-divides by it.
Scope guard: this only affects the displayed composition estimate. It does NOT touch billing, the provider's real prompt_tokens (the separate "Measured occupancy" line), or compression-pressure decisions — those keep the deliberately-conservative char/4 estimate_request_tokens_rough so Hermes still compresses before a provider rejects an oversized payload.
locales/en.yaml: /usage comp_header "char/4 estimate" → "char-based estimate" so the wording survives future tuning.
3 new tests (default-3.5, env override incl. 3.5>4.0 ordering, out-of-range/malformed fallback). 13/13 pass, ruff clean.

Verification

tests/test_request_composition.py 13 passed; ruff clean. Live recompute of 3 real turns shows worst-case delta −15.3% → −3.2%. Companion home-plugin label change ("char/4" → "char-based estimate" in the /context renderer) deployed alongside.

Companion

Builds directly on #21 (request composition snapshot). Independent, can land on its own.

…mate (default 3.5) The /context /usage /compress composition estimate used a flat char/4 rule, which systematically UNDER-counts real Hermes requests (dominated by JSON/code- dense tool results that tokenize closer to ~3.5 chars/token). Measured against live data the delta-vs-measured ran -8% at 170k context and -15% at 280k. Switches the divisor to 3.5 (tightens the live band to ~+-5% / -3.2% at 280k) and makes it tunable via env HERMES_COMPOSITION_CHARS_PER_TOKEN, clamped to a 2.0-8.0 sane range so a typo cannot make the estimate absurd. Only affects the DISPLAYED composition estimate -- never billing, the provider prompt_tokens (Measured occupancy), or compression-pressure decisions (those keep the conservative /4 estimate_request_tokens_rough so Hermes still compresses before a provider rejects the payload). 3 new tests: default-3.5, env override (and 3.5>4.0 ordering), out-of-range fallback. User-facing "char/4" labels -> "char-based estimate" so wording survives future tuning. 13/13 pass, ruff clean.

github-actions · 2026-06-08T23:39:43Z

🔎 Lint report: `feat/composition-chars-per-token` vs `origin/main`

ruff

Total: 0 on HEAD, 0 on base (➖ 0)

🆕 New issues: none

✅ Fixed issues: none

Unchanged: 0 pre-existing issues carried over.

ty (type checker)

Total: 9950 on HEAD, 9950 on base (➖ 0)

🆕 New issues: none

✅ Fixed issues: none

Unchanged: 5156 pre-existing issues carried over.

Diagnostics are surfaced as warnings — this check never fails the build.

…24) PR #23 made the /context /usage /compress composition estimate use the tunable COMPOSITION_CHARS_PER_TOKEN (default 3.5), but the three shared rough estimators (estimate_tokens_rough / estimate_messages_tokens_rough / estimate_request_tokens_rough) still hardcoded char/4. Per Ace: use one variable, 3.5, for everything -- including compression-pressure / pre-flight 413 checks. Adds a single shared _ceil_chars_to_tokens(chars) helper and routes all four //4 sites + compose_request_breakdown._t() through it. char/3.5 estimates HIGHER than char/4, so compression and pre-flight oversize checks fire SOONER, never later -- strictly safer for context-overrun protection, and Ace already runs conservative compression params. Still only a char-based heuristic; the real provider prompt_tokens remains the source of truth for billing/occupancy. Tests updated to the 3.5 divisor (divisor-derived via math.ceil(n/3.5) rather than fresh magic numbers so they survive future tuning). 201 pass, ruff clean. Co-authored-by: Apollo <apollo@daemonarchy.local>

- kanban_db #23: the in-txn takeover audit fires only when the txn changed the target card's guarded columns (status/assignee/priority/session_id), not on any side write (reclaim_refused), so a refused mutation never records or re-homes. No longer reads conn.total_changes (fixes write_txn_busy_retry fakes, CI slice 13). - kanban_db #44: birth home re-resolved at the TOP of the write txn, before the duplicate / parent checks. - kanban_db #70: no own-claim exception by profile name; without a run id only a lapsed run (claim expired, no live owner) can be closed. - checkout_admission #52: read-check-replace under a per-consumer flock. - discord #30/#31: cancellation recorded before fut.done(); a cancelled disconnect waits for the in-flight flush and still runs every flush. - run.py #36: lock held until the worker write finishes on cancel; every write runs in FIFO order (no skipped-but-reported resets). - ci_overflow_integration #27: documented + tested that a secondary-limit 403 arrives as RateLimited(429) and fails closed. New tests, each red against the previous commit.

…iew send-back re-checked under the lock - #23 (PR951): the takeover/override event and comment are written in the guarded mutator's own write txn (only when it changed rows), so a failed audit rolls the mutation back; the chat subscription stays post-commit. - #44 (PR987): create_task re-resolves the parent home inside its write_txn. - #70 (PR1081): request_changes with a claimer and no expected_run_id refuses to close another holder's live review run (unexpired claim or live owner). Tests red on base: test_takeover_audit_is_in_the_mutators_txn, test_parent_rehomed_before_child_insert_is_the_childs_home, test_claimer_without_run_id_cannot_close_another_reviewers_live_run.

- kanban_db #23: the in-txn takeover audit fires only when the txn changed the target card's guarded columns (status/assignee/priority/session_id), not on any side write (reclaim_refused), so a refused mutation never records or re-homes. No longer reads conn.total_changes (fixes write_txn_busy_retry fakes, CI slice 13). - kanban_db #44: birth home re-resolved at the TOP of the write txn, before the duplicate / parent checks. - kanban_db #70: no own-claim exception by profile name; without a run id only a lapsed run (claim expired, no live owner) can be closed. - checkout_admission #52: read-check-replace under a per-consumer flock. - discord #30/#31: cancellation recorded before fut.done(); a cancelled disconnect waits for the in-flight flush and still runs every flush. - run.py #36: lock held until the worker write finishes on cancel; every write runs in FIFO order (no skipped-but-reported resets). - ci_overflow_integration #27: documented + tested that a secondary-limit 403 arrives as RateLimited(429) and fails closed. New tests, each red against the previous commit.

…iew send-back re-checked under the lock - #23 (PR951): the takeover/override event and comment are written in the guarded mutator's own write txn (only when it changed rows), so a failed audit rolls the mutation back; the chat subscription stays post-commit. - #44 (PR987): create_task re-resolves the parent home inside its write_txn. - #70 (PR1081): request_changes with a claimer and no expected_run_id refuses to close another holder's live review run (unexpired claim or live owner). Tests red on base: test_takeover_audit_is_in_the_mutators_txn, test_parent_rehomed_before_child_insert_is_the_childs_home, test_claimer_without_run_id_cannot_close_another_reviewers_live_run.

- kanban_db #23: the in-txn takeover audit fires only when the txn changed the target card's guarded columns (status/assignee/priority/session_id), not on any side write (reclaim_refused), so a refused mutation never records or re-homes. No longer reads conn.total_changes (fixes write_txn_busy_retry fakes, CI slice 13). - kanban_db #44: birth home re-resolved at the TOP of the write txn, before the duplicate / parent checks. - kanban_db #70: no own-claim exception by profile name; without a run id only a lapsed run (claim expired, no live owner) can be closed. - checkout_admission #52: read-check-replace under a per-consumer flock. - discord #30/#31: cancellation recorded before fut.done(); a cancelled disconnect waits for the in-flight flush and still runs every flush. - run.py #36: lock held until the worker write finishes on cancel; every write runs in FIFO order (no skipped-but-reported resets). - ci_overflow_integration #27: documented + tested that a secondary-limit 403 arrives as RateLimited(429) and fails closed. New tests, each red against the previous commit.

…iew send-back re-checked under the lock - #23 (PR951): the takeover/override event and comment are written in the guarded mutator's own write txn (only when it changed rows), so a failed audit rolls the mutation back; the chat subscription stays post-commit. - #44 (PR987): create_task re-resolves the parent home inside its write_txn. - #70 (PR1081): request_changes with a claimer and no expected_run_id refuses to close another holder's live review run (unexpired claim or live owner). Tests red on base: test_takeover_audit_is_in_the_mutators_txn, test_parent_rehomed_before_child_insert_is_the_childs_home, test_claimer_without_run_id_cannot_close_another_reviewers_live_run.

- kanban_db #23: the in-txn takeover audit fires only when the txn changed the target card's guarded columns (status/assignee/priority/session_id), not on any side write (reclaim_refused), so a refused mutation never records or re-homes. No longer reads conn.total_changes (fixes write_txn_busy_retry fakes, CI slice 13). - kanban_db #44: birth home re-resolved at the TOP of the write txn, before the duplicate / parent checks. - kanban_db #70: no own-claim exception by profile name; without a run id only a lapsed run (claim expired, no live owner) can be closed. - checkout_admission #52: read-check-replace under a per-consumer flock. - discord #30/#31: cancellation recorded before fut.done(); a cancelled disconnect waits for the in-flight flush and still runs every flush. - run.py #36: lock held until the worker write finishes on cancel; every write runs in FIFO order (no skipped-but-reported resets). - ci_overflow_integration #27: documented + tested that a secondary-limit 403 arrives as RateLimited(429) and fails closed. New tests, each red against the previous commit.

Kyzcreig merged commit 9d562fd into main Jun 9, 2026
23 checks passed

Kyzcreig deleted the feat/composition-chars-per-token branch June 9, 2026 13:05

Kyzcreig mentioned this pull request Jun 9, 2026

feat(telemetry): unify all rough token estimators on the 3.5 divisor #24

Merged

Kyzcreig mentioned this pull request Jun 9, 2026

feat(telemetry): route 413/stale-call estimator through shared 3.5 divisor #26

Merged

ang-fleet-workers Bot mentioned this pull request Sep 27, 2026

fix(c5): 14 follow-up race instances from the FleetReview retro-backfill (stacked on #1355) #1361

Merged

Navigation Menu

Repository navigation

feat(telemetry): tunable chars-per-token divisor for composition estimate (default 3.5) - #23

feat(telemetry): tunable chars-per-token divisor for composition estimate (default 3.5)#23
Kyzcreig merged 1 commit into
mainfrom
feat/composition-chars-per-token

Kyzcreig commented Jun 8, 2026

Uh oh!

github-actions Bot commented Jun 8, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

Kyzcreig commented Jun 8, 2026

What

Why

How

Verification

Companion

Uh oh!

github-actions Bot commented Jun 8, 2026

🔎 Lint report: feat/composition-chars-per-token vs origin/main

ruff

ty (type checker)

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

🔎 Lint report: `feat/composition-chars-per-token` vs `origin/main`