Skip to content

fix: improve Hermes executive behaviour classification - #9

Merged
nitinteck merged 1 commit into
mainfrom
feature/hermes-executive-behaviour-classification-fix-v1
Jul 29, 2026
Merged

fix: improve Hermes executive behaviour classification#9
nitinteck merged 1 commit into
mainfrom
feature/hermes-executive-behaviour-classification-fix-v1

Conversation

@nitinteck

Copy link
Copy Markdown
Owner

Objective

Improve the Hermes Executive Orchestrator behavioural baseline after the 20-message diagnostic run and real WhatsApp A1/A2/B1 checks.

Reproduced defects

  • Baseline PR test: record Hermes diagnostic behavioural baseline #8 preserved the pre-fix run: 9 pass / 11 fail, 10/20 classification accuracy.
  • Confirmed classifier overmatched practical/advisory connector discussions and undermatched executive status/data-availability questions.
  • Confirmed trace metadata did not expose safe runtime context categories for recent conversation/profile availability.

Root causes

  • Classifier ordering and keyword checks treated connector nouns as executable too broadly.
  • Status/data-availability phrases were missing from deterministic classification.
  • Diagnostic single-turn path did not enforce enabled orchestrator status before running and did not support a one-session behavioural pack run.

Changes

  • Refines classifier precision for ordinary, executive_status, decision_support, planning_request, potentially_executable and unsupported_or_unsafe requests.
  • Adds safe runtime context categories for recent_conversation, persistent_profile, current_request_metadata and deterministic_output.
  • Adds executive response guidance for concise, practical, evidence-aware responses without architecture leakage.
  • Adds local-only behavioural pack diagnostic runner with fail-fast enabled-status verification.
  • Returns safe context counts, evidence refs and latency metadata in diagnostic results.

Safety model

  • No connectors added.
  • No live execution enabled.
  • Potential external-action requests remain blocked before model execution with execution_state=not_executed.
  • Diagnostic path remains local-only with outbound platform delivery disabled.

Validation

  • python -m compileall -q touched files: pass.
  • pytest tests/gateway/test_executive_orchestrator.py tests/hermes_cli/test_executive_orchestrator_cli.py -q: 45 passed.
  • ruff check touched files: pass.
  • ruff format --check touched files: pass.
  • git diff --check: pass.
  • mypy touched invocation is blocked by existing repo-wide typing baseline debt in unrelated imports/stubs and legacy syntax.

Deployment plan

  • Merge after review.
  • Deploy hermes-agent only.
  • Restart hermes-gateway.service only.
  • Verify Orchestrator enabled, execution boundary not_executed, live_execution_enabled=false.
  • Run production local-only behavioural pack and store post-fix comparison artifacts.

@nitinteck
nitinteck merged commit 103ca75 into main Jul 29, 2026
@nitinteck
nitinteck deleted the feature/hermes-executive-behaviour-classification-fix-v1 branch July 29, 2026 17:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant