fix(middleware): complete transactional request transform contract

[Skywind5487]

Closes #117.

Completion claim

The outbound LLM request transformation extension point is complete as one current-main merge unit. This PR uses the merged llm_request middleware seam and makes its replacement chain genuinely transactional: no shallow-copy fallback can expose shared nested request state to a failing callback.

Current-main / prior-art preflight

Pinned during the Aug 19 rebuild:

• fork Skywind5487/hermes-agent:main: 243352e7b8bddc9f33eba1b6506810f8dd88beaa
• upstream NousResearch/hermes-agent:main: 5dd15872a6878a19b9b5478b6968b38f48dd311f

Prior-art classification:

• Merged authority: upstream PR feat(middleware): add adaptive middleware to hermes-agent, consumed by NeMo-Relay NousResearch/hermes-agent#29724 introduced llm_request / llm_execution middleware and the common request/dispatch seam.
• Superseded design: upstream PR feat: add transform_api_request plugin hook for full-payload request transformation NousResearch/hermes-agent#43241 used the historical mutable-by-reference transform_api_request hook and closed unmerged; its issue was later closed in favor of middleware.
• Open/unmerged evidence: upstream PR fix(middleware): chain request rewrites sequentially NousResearch/hermes-agent#73656 identified the request-callback composition bug.
• Current open salvage: upstream PR fix(middleware): preserve original tool requests NousResearch/hermes-agent#82092 supersedes fix(middleware): chain request rewrites sequentially NousResearch/hermes-agent#73656 and still demonstrates sequential isolated request composition, but remains open/unmerged. As of this rebuild its branch is far behind current upstream main, so it is evidence rather than current authority.

This fork PR implements only the LLM request-transform contract required by #117. It does not pull NousResearch#82092's tool/Relay-specific changes into this merge unit.

What changes

Atomic llm_request chain

apply_llm_request_middleware() keeps the transaction boundary inline and explicit:

1. Ensure first-use plugin discovery has run before snapshotting callbacks, so a configured plugin in a fresh process is never mistaken for "no middleware".
2. No callbacks: return the exact original request object unchanged.
3. Deep-copy the complete provider request before any callback can see it.
4. Run callbacks in plugin registration order on isolated current/original copies.
5. Commit only a valid complete {"request": {...}} replacement that can itself be deep-copied.
6. Mutation followed by exception is discarded; later callbacks continue from the last commit.
7. If the original request contains a non-deepcopyable provider object, log the isolation failure, skip request middleware for that call, and dispatch the original unchanged.

There is deliberately no shallow-copy fallback. A shallow dict copy shares nested lists/dicts and cannot uphold rollback.

The transaction stays inline instead of introducing _run_request_chain(kind, payload_key, original_key, ...); there is only one maintained LLM request transaction today. The shared _safe_copy() helper keeps its tolerant deepcopy-with-shallow-fallback behavior for tool-request middleware — this PR does not change tool-request semantics.

Documented boundary

The existing canonical docs/middleware/README.md now documents the atomic composition/failure semantics and the lazy-discovery delivery guarantee. Its existing execution-order section pins the maintained boundary:

1. provider kwargs assembly
2. llm_request
3. pre_api_request
4. llm_execution
5. provider dispatch
6. post/error observers

Focused regression tests

Tests cover only the contracts that can regress here:

• no-listener identity;
• sequential replacements with isolated stable original_request;
• nested mutation + exception rollback;
• non-deepcopyable original request skips callbacks without exposing shared nested state;
• non-deepcopyable replacement is rejected before commit;
• fresh-process lazy discovery: a configured llm_request plugin runs on first delivery even before discovery has run (calls apply_llm_request_middleware() without an explicit discover_and_load());
• transform remains before observers and the common provider dispatch/execution seam.

PR topology

This PR was rebuilt from current fork main, not the old Phase-2 dev lineage.

• base / merge-base: 243352e7b8bddc9f33eba1b6506810f8dd88beaa
• branch: fork/request-transform
• commits (intent history): 13c3f787a transactional llm_request request transform; a9739e3a2 preserve lazy plugin discovery for llm_request
• relative to main: ahead 2 / behind 0
• changed files: hermes_cli/middleware.py, tests/hermes_cli/test_request_transform_contract.py, tests/hermes_cli/test_plugins.py, docs/middleware/README.md

Acceptance mapping

• Complete outbound provider request is exposed at a documented boundary.
• Request-transform ordering is explicit and regression-tested.
• Disabled/missing middleware preserves the exact original request object.
• Callback failure cannot dispatch partially transformed request data.
• Non-deepcopyable request payloads fail open without shallow shared-state exposure.
• Provider coverage is pinned to the common dispatch seam rather than one adapter path.
• One ticket → one current-main branch → one PR.

[github-actions]

૮ >ﻌ< ა ci review

_{ran on 0517125 — docs(middleware): document atomic LLM request transforms}

⚠️ Warnings

OSV vulnerability scan · View job

5 known vulnerabilities found in pinned dependencies.

• CVE-2026-67213 in package-lock.json
• CVE-2026-67213 in website/package-lock.json
• CVE-2026-70608 in package-lock.json
• CVE-2026-56876 in package-lock.json
• CVE-2026-70606 in package-lock.json

How to fix:

Review the findings in the Security tab. Update the affected dependencies if a patched version is available.

---

debug info

CI timings

CI timings · View report · View job

Wall time 2m58s vs 3m25s (-13.2%). 9 job(s) slower, 13 faster, 2 unchanged.

• Python tests / Run tests slice 10/12: -32.0s
• Python tests / Run tests slice 3/12: -19.0s
• Python tests / Run tests slice 6/12: -11.0s
• Python tests / Run tests slice 1/12: -11.0s
• Python tests / Run tests slice 7/12: -10.0s

[Skywind5487]

Validation update — Aug 19 current-main rebuild

The Aug 17 dev-based validation is superseded. Current PR head: 05171254cc83c2fe8099fcc00d118110ede43acf.

Topology

• base / merge-base: 243352e7b8bddc9f33eba1b6506810f8dd88beaa (Skywind5487/hermes-agent:main)
• relative to main: ahead 3 / behind 0
• changed files: only hermes_cli/middleware.py, tests/hermes_cli/test_request_transform_contract.py, and docs/middleware/README.md

Correctness blocker

The shallow-copy fallback is gone. llm_request callbacks now receive only fully deep-copied candidates. If the original request cannot be deep-copied, middleware is skipped and the exact original request is dispatched unchanged; if a returned replacement cannot be deep-copied, that replacement is rejected before commit. Mutation-before-raise therefore cannot leak through the previous shallow-copy path.

The generic _run_request_chain(...) abstraction is also gone; the transaction boundary stays inline in apply_llm_request_middleware() until a second real request-chain use case exists.

Final-head CI

Head 0517125...:

• CI run 32213359352: completed / success
• Docker Build, Test, and Publish run 32213358749: completed / success
• full-tree ruff enforcement: success
• Windows footguns: success
• PR-relative ruff + ty diff: success
• Python e2e: success
• macOS-only tests: success
• Windows-only tests: success
• attribution / common-ancestor / supply-chain / OSV: success
• all 12 Python test slices: success

Focused regression evidence from slice 4/12:

tests/hermes_cli/test_request_transform_contract.py (6✓, 2.4s)

That file includes the non-deepcopyable-provider-object regression and nested mutation-before-raise rollback regression, so the two correctness boundaries are exercised on the final head rather than inferred from aggregate CI status.

Ready for re-review; not merged.

[Skywind5487]

HOLD — two blocking contract regressions found. Full two-axis report is on #117.

1. apply_llm_request_middleware() switched from discovery-aware has_middleware() / invoke_middleware() to _get_middleware_callbacks() → get_plugin_manager()._middleware. The plugin delivery layer explicitly lazy-discovers on first middleware delivery; the raw registry lookup does not. A configured llm_request plugin can therefore look “missing” in a fresh process and the request will be dispatched unchanged. Please preserve a discovery-aware gate/snapshot and regression-test first-use delivery without an explicit discover_and_load().

2. The _safe_copy() change is outside this LLM-only merge unit and changes tool-request behavior. apply_tool_request_middleware() shares that helper, so non-deepcopyable tool args that previously degraded to a shallow dict copy can now raise. The LLM transaction already uses direct deepcopy() locally, so the smallest safe fix is to leave _safe_copy() at base semantics and keep strict isolation inside apply_llm_request_middleware() only.

CI is green and the actual transaction semantics (ordered commits, stable original snapshot, rollback after callback mutation/failure, no-listener identity, common dispatch seam) otherwise look solid.