Skip to content

fix(honcho): debounce transient dream contention - #206

Closed
batumilove wants to merge 1 commit into
batumi/livefrom
fix/honcho-dream-contention-debounce-20260811
Closed

batumilove wants to merge 1 commit into
batumi/livefrom
fix/honcho-dream-contention-debounce-20260811

Conversation

@batumilove

@batumilove batumilove commented Aug 11, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • render successful but degraded Spark chat probes as amber
  • omit the misleading recent-errors line when both counters are zero
  • debounce expected fresh-Dream contention until two consecutive degraded samples
  • preserve immediate alerts for critical latency (30s), stale Dream work, non-Dream degradation, model failure, and thinking responses

Evidence

  • strict TDD: new contract tests failed before implementation
  • venv/bin/python -m pytest tests/hermes_cli/test_honcho_monitor.py -o 'addopts=' -q — 55 passed
  • venv/bin/ruff check hermes_cli/honcho_monitor.py tests/hermes_cli/test_honcho_monitor.py — passed
  • Python compile and git diff --check — passed
  • live read-only monitor smoke — nominal; 1536-dimensional DB inventory intact
  • independent bounded review — PASS after fixing its finding that thinking responses must not increment the Dream-contention streak

Scope

Two files only. No service restart, queue mutation, configuration change, or upstream-project PR.

Summary by CodeRabbit

  • New Features

    • Added clearer spark-goat latency statuses for normal, degraded, and critical conditions.
    • Added tracking for recurring dream-contention conditions across monitoring runs.
    • Alerts now distinguish temporary degraded latency from sustained contention.
  • Bug Fixes

    • Reduced false alarms by requiring repeated degraded contention before alerting.
    • Critical latency conditions continue to trigger alerts immediately.
    • Reports now show recent error counts only when errors are present.
    • Degraded or thinking states are displayed with an orange status indicator.

@github-actions

Copy link
Copy Markdown

૮ >ﻌ< ა ci review

running on bc83249 — fix(honcho): debounce transient dream contention

waiting for jobs to start…

@coderabbitai

coderabbitai Bot commented Aug 11, 2026 •

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

The monitor now distinguishes degraded and critical chat latency, debounces dream-contention alerts with a persisted streak, and updates report status and error rendering. Tests cover alert timing, streak resets, degraded status, and zero-error output.

Changes

Honcho monitor alert handling

Layer / File(s) Summary
Contention streak calculation
hermes_cli/honcho_monitor.py, tests/hermes_cli/test_honcho_monitor.py
The monitor adds degraded and critical latency thresholds. It tracks fresh, non-thinking dream contention and advances or resets the persisted streak.
Gated alerts and report state
hermes_cli/honcho_monitor.py, tests/hermes_cli/test_honcho_monitor.py
Contention alerts require critical latency or a previous streak. The monitor persists the streak and updates degraded status and recent-error rendering.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant MonitorSnapshot
  participant StreakCalculator
  participant AlertEvaluator
  participant MonitorState
  MonitorSnapshot->>StreakCalculator: provide fresh dream contention sample
  StreakCalculator->>MonitorState: calculate and persist contention streak
  MonitorSnapshot->>AlertEvaluator: provide latency and contention state
  AlertEvaluator->>AlertEvaluator: emit alert for critical latency or prior streak
  AlertEvaluator->>MonitorState: update alert and report state
Loading
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: debouncing transient Dream contention alerts in Honcho monitoring.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/honcho-dream-contention-debounce-20260811

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@hermes_cli/honcho_monitor.py`:
- Line 1149: Update the state-update flow around next_dream_contention_streak so
ssh-unavailable samples retain previous_state["dream_contention_streak"] instead
of resetting it to zero, while keeping normal probe-based updates unchanged when
SSH is available. Extend the runtime tests to cover two degraded samples
separated by one SSH-unavailable sample and verify the contention alert remains
correctly debounced.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 3bafb8cf-79f5-4e65-9b30-83b506279495

📥 Commits

Reviewing files that changed from the base of the PR and between 74e7884 and bc83249.

📒 Files selected for processing (2)
  • hermes_cli/honcho_monitor.py
  • tests/hermes_cli/test_honcho_monitor.py

def main() -> int:
snapshot, current_state = collect_snapshot()
previous_state = load_state(STATE_PATH)
current_state["dream_contention_streak"] = next_dream_contention_streak(snapshot, previous_state)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -e
printf '%s\n' '--- file outline ---'
ast-grep outline hermes_cli/honcho_monitor.py
printf '%s\n' '--- target region ---'
sed -n '1040,1185p' hermes_cli/honcho_monitor.py
printf '%s\n' '--- related identifiers ---'
rg -n -C 5 'dream_contention_streak|next_dream_contention_streak|ssh_ok|valid Spark|spark' hermes_cli/honcho_monitor.py
printf '%s\n' '--- relevant tests and scripts ---'
rg -n 'dream_contention_streak|next_dream_contention_streak|honcho_monitor|run_tests' . --glob '!*.lock' --glob '!node_modules/**'

Repository: batumilove/hermes-agent

Length of output: 46564


🏁 Script executed:

#!/bin/bash
set -e
printf '%s\n' '--- state restoration helpers ---'
sed -n '140,255p' hermes_cli/honcho_monitor.py
printf '%s\n' '--- contention tests ---'
sed -n '320,410p' tests/hermes_cli/test_honcho_monitor.py
printf '%s\n' '--- repository refs ---'
git branch --all --no-color
git status --short
git diff --stat main...HEAD -- hermes_cli/honcho_monitor.py tests/hermes_cli/test_honcho_monitor.py 2>/dev/null || true
printf '%s\n' '--- main implementation ---'
git show main:hermes_cli/honcho_monitor.py 2>/dev/null | sed -n '280,390p;1135,1175p' || true
printf '%s\n' '--- current implementation references ---'
rg -n -C 8 'preserve_last_valid_probe_state|restore_last_valid_remote_sample|dream_contention_streak|ssh_failure_streak' hermes_cli/honcho_monitor.py tests/hermes_cli/test_honcho_monitor.py

Repository: batumilove/hermes-agent

Length of output: 27163


🏁 Script executed:

#!/bin/bash
set -e
printf '%s\n' '--- snapshot collection and SSH failure construction ---'
sed -n '925,1045p' hermes_cli/honcho_monitor.py
sed -n '825,845p' hermes_cli/honcho_monitor.py
sed -n '835,925p' tests/hermes_cli/test_honcho_monitor.py
printf '%s\n' '--- commit and ref relationships ---'
git rev-parse HEAD
git rev-parse HEAD^
git rev-parse --verify origin/batumi/live
git diff --stat HEAD^ HEAD -- hermes_cli/honcho_monitor.py tests/hermes_cli/test_honcho_monitor.py
git diff HEAD^ HEAD -- hermes_cli/honcho_monitor.py tests/hermes_cli/test_honcho_monitor.py | sed -n '1,260p'
printf '%s\n' '--- parent implementation ---'
git show HEAD^:hermes_cli/honcho_monitor.py 2>/dev/null | sed -n '280,390p;1135,1175p' || true

Repository: batumilove/hermes-agent

Length of output: 24120


🏁 Script executed:

#!/bin/bash
set -e
printf '%s\n' '--- SSH wrapper and active-probe parsing ---'
sed -n '680,735p' hermes_cli/honcho_monitor.py
sed -n '1020,1140p' hermes_cli/honcho_monitor.py
printf '%s\n' '--- standalone sequence verifier ---'
python3 - <<'PY'
from dataclasses import dataclass

DEGRADED = 5.0
CRITICAL = 30.0
STALE = 1800

`@dataclass`
class Snapshot:
    ssh_ok: bool
    spark_ok: bool
    latency: float
    thinking: bool
    active_dream_count: int
    active_dream_age: int

def fresh_contention(s):
    return (
        s.spark_ok and not s.thinking and s.latency > DEGRADED
        and s.active_dream_count > 0 and s.active_dream_age < STALE
    )

def next_streak(s, previous):
    return previous + 1 if fresh_contention(s) else 0

def contention_alert(s, previous):
    if not s.ssh_ok:
        return False
    if not s.spark_ok or s.thinking or s.latency <= DEGRADED:
        return False
    if s.active_dream_count <= 0 or s.active_dream_age >= STALE:
        return False
    return s.latency >= CRITICAL or previous >= 1

healthy_degraded = Snapshot(True, True, 12.5, False, 1, 60)
ssh_unavailable = Snapshot(False, True, 1.2, False, 0, 0)

state = 0
state = next_streak(healthy_degraded, state)
print("first degraded sample:", state, "alert:", contention_alert(healthy_degraded, 0))

state = next_streak(ssh_unavailable, state)
print("SSH-unavailable sample:", state, "alert:", contention_alert(ssh_unavailable, 1))

alert = contention_alert(healthy_degraded, state)
state = next_streak(healthy_degraded, state)
print("second degraded sample:", state, "alert:", alert)

preserved = 1
print("second degraded sample with preserved streak:",
      contention_alert(healthy_degraded, preserved),
      "next state:", next_streak(healthy_degraded, preserved))
PY
printf '%s\n' '--- sub-probe error markers and state fields ---'
rg -n -C 3 '__SSH_ERROR__|db_probe_ok|active_raw|active_dream_count|services = _parse_service_status' hermes_cli/honcho_monitor.py

Repository: batumilove/hermes-agent

Length of output: 15669


Preserve dream_contention_streak across SSH-unavailable samples.

When ssh_ok is false, the missing Dream probe makes next_dream_contention_streak() return 0. The fallback restores report fields only. A later degraded sample is then debounced again, suppressing its contention alert. Preserve the previous streak and add a runtime test for two degraded samples separated by one SSH-unavailable sample.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@hermes_cli/honcho_monitor.py` at line 1149, Update the state-update flow
around next_dream_contention_streak so ssh-unavailable samples retain
previous_state["dream_contention_streak"] instead of resetting it to zero, while
keeping normal probe-based updates unchanged when SSH is available. Extend the
runtime tests to cover two degraded samples separated by one SSH-unavailable
sample and verify the contention alert remains correctly debounced.

Source: Coding guidelines

@batumilove

Copy link
Copy Markdown
Owner Author

Implemented by merged upstream-sync PR #209 and present unchanged on current live commit f966629. Both PR-touched file blobs match bc83249 exactly; replaying the PR commit onto live is empty. Fresh isolated verification on live: 55 focused tests passed, Ruff/py_compile clean. Closing without merge as already integrated.

@batumilove batumilove closed this Aug 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant