test: reorganize test structure and add missing unit tests by 0xbyt4 · Pull Request #34 · NousResearch/hermes-agent

0xbyt4 · 2026-02-26T00:27:59Z

Summary

Reorganize flat tests/ directory to mirror source code structure (tools/, gateway/, hermes_cli/, integration/)
Add 11 new test files covering previously untested modules
Add pytest integration marker to gate external-service tests

New Test Architecture

tests/
├── conftest.py                        # Shared fixtures (tmp_dir, mock_config)
│
├── tools/                             # mirrors tools/ source
│   ├── test_registry.py               # NEW  - 10 tests  ← registry.py
│   ├── test_patch_parser.py           # NEW  -  8 tests  ← patch_parser.py
│   ├── test_fuzzy_match.py            # NEW  -  9 tests  ← fuzzy_match.py
│   ├── test_todo_tool.py              # NEW  - 11 tests  ← todo_tool.py
│   ├── test_approval.py               # NEW  - 14 tests  ← approval.py
│   ├── test_file_tools.py             # NEW  -  9 tests  ← file_tools.py
│   ├── test_code_execution.py         # MOVED - 18 tests
│   ├── test_delegate.py               # MOVED - 22 tests
│   └── test_interrupt.py              # MOVED -  6 tests
│                                                107 unit tests
│
├── gateway/                           # mirrors gateway/ source
│   ├── test_session.py                # NEW  -  6 tests  ← session.py
│   ├── test_config.py                 # NEW  -  8 tests  ← config.py
│   └── test_delivery.py               # NEW  - 15 tests  ← delivery.py
│                                                 29 unit tests
│
├── hermes_cli/                        # mirrors hermes_cli/ source
│   ├── test_config.py                 # NEW  -  6 tests  ← config.py
│   └── test_models.py                 # NEW  -  5 tests  ← models.py
│                                                 11 unit tests
│
└── integration/                       # tests requiring external services
    ├── test_web_tools.py              # MOVED (requires FIRECRAWL_API_KEY)
    ├── test_modal_terminal.py         # MOVED (requires Modal backend)
    ├── test_batch_runner.py           # MOVED (manual script)
    └── test_checkpoint_resumption.py  # MOVED (requires batch_runner)

Before: 7 test files in flat tests/, ~86 tests, ~10% coverage
After: 18 test files in organized structure, 147 unit tests + 14 integration tests

What changed

Change	Files
New test files	11 (conftest + 6 tools + 3 gateway + 2 hermes_cli)
Moved test files	7 (3 → tools/, 4 → integration/)
New `__init__.py`	4 (tools, gateway, hermes_cli, integration)
Updated	pyproject.toml (pytest markers + addopts)

Running tests

# Unit tests only (default, excludes integration)
uv run pytest -v

# Specific module
uv run pytest tests/tools/ -v
uv run pytest tests/gateway/ -v
uv run pytest tests/hermes_cli/ -v

# Integration tests (requires API keys / external services)
uv run pytest -m integration -v

# Everything
uv run pytest -m '' -v

Test plan

uv run pytest tests/tools/ -v — 107 passed
uv run pytest tests/gateway/ -v — 29 passed
uv run pytest tests/hermes_cli/ -v — 11 passed
uv run pytest -v — 147 passed, 9 deselected (integration)
uv run pytest -m integration --collect-only — 9 collected correctly

Reorganize flat tests/ directory to mirror source code structure (tools/, gateway/, hermes_cli/, integration/). Add 11 new test files covering previously untested modules: registry, patch_parser, fuzzy_match, todo_tool, approval, file_tools, gateway session/config/ delivery, and hermes_cli config/models. Total: 147 unit tests passing, 9 integration tests gated behind pytest marker.

teknium1 · 2026-02-26T07:49:28Z

LGTM!

…ix (NousResearch#34) Squash merge of feat/mobile-setup-and-fixes

…legram notification - _spawn_cc: catch TimeoutExpired, return timed_out=True - _notify_failure: haiku summarises stdout, sends Telegram via home channel - _action_address_comments: detect timeout/exit!=0/blocked, call _notify_failure Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

Doc-parity with linxule/hermes-kimi-plugin's NousResearch#34 docstring expansion at the same init site. Two-path architecture (adapter.send() vs the module- level send_kimi_message helper) made explicit; when-to-use guidance for group rooms vs DMs; failure modes when the agent isn't system-prompted to call send_message_tool. Comment-only — no runtime behaviour change. Keeps this legacy variant useful as an inspection ref alongside the standalone plugin. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…d-unit-tests test: reorganize test structure and add missing unit tests

- path-to-regexp: 0.1.12 → 0.1.13 (CVE-2026-4867, GHSA-37ch-88jc-xwx2) - picomatch: 2.3.1 → 2.3.2 (CVE-2026-33671, GHSA-c2c7-rcm5-vvqj) - undici: 7.23.0 → 7.27.1 (CVE-2026-1526/CVE-2026-2229/CVE-2026-1528) - serialize-javascript: 6.0.2 → 7.0.3 (GHSA-5c6j-r48x-rmvq, via overrides) Fixes Dependabot alerts #37, #34, #33, #32, #29, #27

…tdin Deploy NousResearch#34: the lock itself engaged (allow_anon=false, hermes key loaded, confirmed by the /health dump in the failure diagnostics), but the post-flip gate probe ran 'docker exec' without -i — the heredoc never reached python3's stdin, the probe printed nothing, and the empty output failed the LOCK_OK check, falsely redding an otherwise successful cutover. The wire step already used -i; the gate now does too, so the 401/keyed-auth probes actually run. https://claude.ai/code/session_018vp1LDdpoaRLCNZQRsnAp8

…arch#34, NousResearch#65) - NousResearch#33 GH_TOKEN propagation: _inject_gh_token_into_env runs `gh auth token` before each worker subprocess.Popen and injects into env if the worker doesn't have GH_TOKEN/GITHUB_TOKEN. - NousResearch#34 respawn_guarded active_pr exempts review roles (tony/tchalla/ vision/reviewer). Bounded: exemption only applies while consecutive_failures < max_retries; once exhausted the guard fires. - NousResearch#65 fabricated github-auth block claims rejected: FabricatedAuthClaimError raised when kanban_block reason matches auth-claim pattern AND dispatcher's `gh auth status` succeeds. Strict leading-position regex with cause:/blocker:/reason:/infra: prefix exemption. Context: hermes-jarvis#61. 33 tests pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…osing-Tag-Breakout-Klasse) _maybe_wrap_untrusted interpolierte content UND source="{name}" ROH -> ein literaler </untrusted_tool_result> im Payload (break-the-wrap) ODER ein geforgter Opener brach aus dem Rahmen aus; das Brain las die nachfolgende Injektion als Anweisung. Auf TOOLS-faehigen Lanes (web_extract/web_search/browser_*/mcp_*) = Actuation-Risiko (untrusted-Voice/SMS sind no_mcp, daher dort max. Sprachausgabe-Kaperung). Fix (evasion-fest, KEIN Token-Matching): - neuer Helper _escape_untrusted_markup: &,<,> entity-encoden, & ZUERST (kein Double-Encode). - content UND name werden vor Interpolation escaped; der name wird zusaetzlich via str.splitlines() auf EINE Zeile kollabiert (voller Unicode-Zeilentrenner-Satz: LF/CR/VT/FF/ FS/GS/RS/NEL/U+2028/U+2029 -- nicht nur LF/CR), damit ein hostiler mcp_<server>-Name die einzeilige Opening-Tag-Zeile nicht brechen kann. - Re-Entrancy-Skip ENTFERNT: der Check las einen fälschbaren attacker-kontrollierten Prefix (content startswith "<untrusted_tool_result") und war selbst ein Skip-the-Wrap-Evasions- Vektor. Keine Reihenfolge macht ihn lebendig UND sicher. Forwards werden jetzt escaped+ re-wrapped (innere Tags inert) -- sicher, da KEIN Consumer den Wrapper parst/strippt (grep-verifiziert: einzige Referenz war die Funktion selbst). Tests: test_does_not_double_wrap -> test_already_wrapped_content_is_escaped_not_skipped; +4 (literal-closer-counter==1, source-name-markup-escape, source-name-unicode-linebreaks, ampersand-first-no-double-encode). 31 passed; Revert-Check beweist alle Escape-Tests fallen bei Identity-Escape bzw. \n\r-only-Strip. 2 adversariale Lenses: breakout_closed=closed. Pre-existing Residuen NICHT in NousResearch#34 gefixt (bewusst): <32-char-skip (kurze Injektion ungeframed), multimodal-Listen-Passthrough (Text-Parts nicht escaped). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…bewahrt), inert Schritt 2 / Engine-Seite (Branch jarvis/engine-deploy). Das tool-lose Brain (no_mcp) hat nur EINEN Ausgabekanal: Text. Der api-server (NICHT das Brain, NICHT die Bridge) durchsucht den Antworttext nach einem festen Marker-Block (<<<JARVIS_ACTION_INTENT>>>...<<<END...>>>), trennt ALLE Bloecke vom TTS-content (kein Markup wird vorgelesen) und haengt sie als additives Antwortfeld action_intent = LISTE der je-Block JSON-geparsten Kandidaten an. Das Brain bleibt 0-Tools (kein neues Tool) -- die neue Naht aus Konzept §0.1 (media_instruction war TOOL-emittiert und nicht wiederverwendbar). - gateway/platforms/api_server.py: extract_action_intent() (fail-soft: wirft NIE; kein Marker -> ("", None)/unveraendert; Block-Cap 8 gegen DoS; nicht-JSON-Block wird NICHT geraten, aber dennoch aus dem TTS gestrippt) + Wiring in _handle_session_chat (additives Feld am hermes.session.chat.completion-Body). - BEWUSST keine Validierung/Zaehlung in der Engine: Engine extrahiert nur (kennt die Lane nicht); die 0/1/>1-DROP + Schema + Lane-Whitelist sind im privilegierten Gate (evaluate_intent ist fuer die Liste gebaut). Extraktion ist NICHT die Sicherheitsgrenze -> ein geforgter Intent erzeugt nur einen Kandidaten, den Gate + menschliche Out-of-Band-Bestaetigung abfangen (kein NousResearch#34-Wiedergaenger). - Marker-Vertrag SSOT = ops/contracts/jarvis_action_intent_schema.yaml (3-Weg-Kopplung Engine/Persona/Schema). Tests: tests/gateway/test_session_chat_action_intent.py (10) + test_session_api (12 Regression) = 22 passed im Image jarvis-hermes-engine:phase2-dc15b3ddf (selbst verifiziert). KEIN Rebuild/Deploy (inert bis Engine-Rebuild). Pre-Live offen: _handle_session_chat_stream strippt Marker noch nicht (von der Bridge nicht genutzt); roher Block re-entert conversation_history vor dem Strip (state.db), single-user akzeptabel, vor Live bewerten. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…d-unit-tests test: reorganize test structure and add missing unit tests

VaultStore.recall() + vault_shadow_recall() + memory(action='recall'), flag-gated (recall_enabled, default aus), dark. tsvector-first -- exakt-KNN ist inert bis DLP+Embedding+Zone 52 (jede confirmed Zeile hat embedding=NULL). - Vollständiger WHERE-Kontrakt (verlustfreie Inverse des Write-Provenienz- Pfads): lifecycle_status='confirmed' AND deleted_at/quarantined_at/ superseded_at IS NULL AND tsvector-Match; tenant/owner via RLS-GUCs, NICHT im Query-Text. - Untrusted-Wrap am Modell-Rand (entity-encode &,<,>): recalled Text ist DATEN, keine Anweisung -- Security-Vokabular bleibt abrufbar (NousResearch#75 warn-vs- block), Wrap-Escape unmöglich (Task NousResearch#34). Kein sanitization_state-Filter. - Fail-soft + Ehrlichkeits-Klausel: leer != Abwesenheit; Fehler/tote DB -> available=False, NIE als "kein Gedächtnis". - Read-only unter vault_transaction (RLS), fetchall innerhalb der Txn. Beweis: 197 Unit-Tests grün (WHERE-Korrektheit, RLS-GUC-Reihenfolge, Wrap- Escape, fail-soft, Ehrlichkeit). recall_enabled bleibt aus (kein Deploy). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…s-Merge, Differenzierer erhalten Standard-3-Wege-Merge (KEIN Rebase/Force): Eltern[1]=21b01cb50 (Prä-Update-HEAD), Eltern[2]=v2026.7.7.2 (b7751df, Upstream-Release "chore: release v0.18.2"). fork/main bewegt sich nur vorwärts -> Codex zieht per FF. Baseline gepinnt auf den Release-Tag, nicht den driftenden origin/main. Mechanik/Analyse: docs/llm/2026-07-11_PLAN_hermes-018-migration.md. WARUM Standard-Merge statt M3-read-tree: _config_version 30->33 trägt Upstream-Migrations- Funktionen; ein Wholesale-checkout hätte die lautlos verworfen. merge-tree: 60/78 auto-merge, 18 echte Konflikte (via Workflow aufgelöst + adversarial verifiziert), 41 our-only unverändert. DIFFERENZIERER ERHALTEN (Vollständigkeits-Gate + Symbol/Marker-Survival-Sweep: 100 Symbole + 35 Marker überleben): Vault (tools/vault/*, 9 Dateien), trusted_surface, stt_redaction, research_enqueue_worker, threat_patterns (4-Tupel + DE-Muster), wrap-escape (NousResearch#34), untrusted_origin (kanban Deprivilege), Recall-Passthrough (memory/tool_executor/runtime_helpers). SICHERHEITS-KORREKTUREN (zwei echte Silent-Drops, vom Testlauf gefangen — der Resolve-Workflow + adversariale Verifikation sahen sie NICHT): * no_mcp-Boden (hermes_cli/tools_config.py): der blosse Config-Pin `api_server: []` GENÜGT in 0.18.2 NICHT — ohne no_mcp-Sentinel schleust _get_platform_tools alle MCP-Server + (bewiesen) 16 Toolsets inkl. computer_use/code_execution/browser auf den untrusted Voice-Pfad (RCE). Fix: In-Code-Output-Backstop `if platform in FAIL_CLOSED_PLATFORMS: return set()` am Funktions-Ausgang (churn-robust). ADR-0039 Invariante 1. Zur Laufzeit bewiesen. * _kanban_worker_skill_available (hermes_cli/kanban_db.py): Merge droppte Funktion + Aufrufstelle (upstream refaktorierte drumherum). Restauriert (Def + Call vor `if task.skills:` in _default_spawn). 32 Deprivilege-Tests wieder grün. BEWUSSTE DROPS (Fork-Lock-Reduktion): §8b-Vault-Seed aus DEFAULT_CONFIG (Kern-Edit -> gehört in externes config_file, Fabel-5); no_mcp-Kern-Clamp (ersetzt durch Output-Backstop). BEWUSSTE DIVERGENZEN / QUARANTÄNEN (test-only, jede dokumentiert; Produktionscode unberührt): write_approval NousResearch#46783 (unser origin-konditionales Gate vs Upstreams unbedingtes Staging -> approval.py-Reconciliation deferred an Fabel-5); api_server-Boden-Tests (Upstream nimmt api_server=tool-tragend an, wir klemmen ihn — 2 skip + 1 zur Regressionswache adaptiert); reasoning_effort-Forwarding (schon bei HEAD deaktiviert, e57d515/NousResearch#60 Codex-Stream-Stall); kanban-subscribe Test-Isolations-Leak (reset_session_vars); cron-ticker->housekeeping-Refactor; stale -worktree-Pfad -> runtime/hermes-main; hook-handler skip-if-absent (externes Deploy-Artefakt). VERIFIKATION: 1578 passed / 32 skipped (betroffene Bereiche: §6-Drift-Stellen, Vault, alle Differenzierer, Touched-Files). uv.lock via `uv lock` regeneriert (psycopg/asyncpg/cryptography). DEFERRED (nicht "in the tree"-blockierend): ruff/ty-Lauf; Externalisierung der Sicherheits- Schichten auf native Hooks; Bahn-A-Mount der hochpriv-Tools (Fabel-5-Aktivierung). Rollback: Tag backup/pre-018 @ 21b01cb; Vorwärts `git revert -m 1 <merge-commit>` (NIE reset). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

… Phase 1) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…mic router Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…ference (NousResearch#34) Fulfills the docs follow-up from NousResearch#32 (§1 plugin loader) — the module docstring in artifact_actions.py already points at docs/plugins/actions.md for the entity-ref wrong-vs-right example; this adds the file. Covers: - Where plugins live, handler signature, register_handler contract - Explicit reload (RPC / chat / startup), staged swap semantics, registry diff + ledger pairing - The authorship/activation split and why reload is never file-watched - The mandatory entity-ref resolution rule with wrong-vs-right code - Destructive role declaration and the challenge flow - Full reference plugin: linear.issue.delete — lookup from pinned content, linear_id from stored row fields, env-var credential, GraphQL error surfacing - Artifact binding declaration + inline HTML marker wiring - Iteration loop and built-in override behavior Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

…rch#34) TestBuildSSHCommand and TestControlSocketPath patched subprocess.run and subprocess.Popen but not shutil.which, so _ensure_ssh_available() raised RuntimeError on CI runners (and any env) where the OpenSSH client is not installed. Add monkeypatch.setattr(ssh_env.shutil, 'which', ...) to both _mock_connection fixtures so the 7 affected tests pass without requiring a real ssh binary.

…nd file the 47 pre-existing failures (NousResearch#89) The ledger's headline number was four committed slices stale (62.1% for conversation_loop, which now measures 99.33%). Replaced guesswork with a full measured run. Backend baseline: 64.09% statements (178,713/268,279), branches 56,530/98,800. Full suite: 2834 files, 29,419 tests passed, 47 failed, 950s at 8 workers, 3.4GB peak. tests/agent (534 files / 7428 tests) and tests/run_agent (190 files / 1708 tests) are both 100% green; all 47 failures live in tests/plugins, tests/tools and two root files, and are pre-existing — spot-checked two that are a repo-hygiene lint tripping on the gitignored .hermes-sandbox artifact dir and an unrelated regex test. Filed as NousResearch#89 rather than left as ambient red. Also corrected the measurement recipe in the worklist: --coverage is NOT an argparse option on run_tests_parallel.py (it is passed through to pytest, which rejects it and reports every file as failed); the only switch is SIJAV_TEST_COVERAGE=1, and coverage must be installed IN THE VENV because the runner spawns subprocesses with the venv interpreter. Recorded the lowest-coverage large modules as the NousResearch#35 roadmap.

…arch#14 NousResearch#18 NousResearch#34-3a)

teknium1 merged commit dcf370c into NousResearch:main Feb 26, 2026

h4x3rotab pushed a commit to Clawdi-AI/hermes-agent that referenced this pull request Apr 10, 2026

feat: mobile setup wizard, paste fix, CLI kill fix, OpenClaw update f…

05e9059

…ix (NousResearch#34) Squash merge of feat/mobile-setup-and-fixes

angelburgosrosado pushed a commit to angelburgosrosado/hermes-agent that referenced this pull request Apr 27, 2026

Merge pull request NousResearch#34 from 0xbyt4/test/reorganize-and-ad…

f848bbd

…d-unit-tests test: reorganize test structure and add missing unit tests

acoastalfog mentioned this pull request Jun 3, 2026

Guided KB Review Sessions: Telegram queue cards #38025

Closed

dizhaky mentioned this pull request Jun 4, 2026

fix(deps): resolve HIGH severity npm vulnerabilities in website dizhaky/hermes-agent#3

Merged

Kyzcreig mentioned this pull request Jun 21, 2026

fix(discord): stop "is typing…" orphaned by typing-loop recreate race #49924

Closed

waefrebeorn pushed a commit to waefrebeorn/slermes that referenced this pull request Jul 2, 2026

Merge pull request NousResearch#34 from 0xbyt4/test/reorganize-and-ad…

d8b7b36

…d-unit-tests test: reorganize test structure and add missing unit tests

Soju06 added a commit to Soju06/hermes-agent that referenced this pull request Jul 16, 2026

docs(patches): register model-routing (patch NousResearch#34, ADR-003…

1bc60c1

… Phase 1) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Soju06 added a commit to Soju06/hermes-agent that referenced this pull request Jul 16, 2026

docs(patches): update NousResearch#34 model-routing with Phase 2 dyna…

82a6e81

…mic router Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

choi138 added a commit to choi138/hermes-agent that referenced this pull request Aug 18, 2026

merge: integrate M5 runtime-control chain (#1 NousResearch#8 NousRese…

0b306d3

…arch#14 NousResearch#18 NousResearch#34-3a)

Navigation Menu

Uh oh!

test: reorganize test structure and add missing unit tests - #34

test: reorganize test structure and add missing unit tests#34
teknium1 merged 1 commit into
NousResearch:mainfrom
0xbyt4:test/reorganize-and-add-unit-tests

0xbyt4 commented Feb 26, 2026

Uh oh!

teknium1 commented Feb 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

0xbyt4 commented Feb 26, 2026

Summary

New Test Architecture

What changed

Running tests

Test plan

Uh oh!

teknium1 commented Feb 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants