Skip to content

fix(nemo-relay): harden native and CLI integration - #65104

Draft
bbednarski9 wants to merge 2 commits into
NousResearch:mainfrom
bbednarski9:bbednarski/nemo-relay-managed-codecs
Draft

fix(nemo-relay): harden native and CLI integration#65104
bbednarski9 wants to merge 2 commits into
NousResearch:mainfrom
bbednarski9:bbednarski/nemo-relay-managed-codecs

Conversation

@bbednarski9

@bbednarski9 bbednarski9 commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Summary

  • select NeMo Relay's public request/response codec from Hermes' resolved api_mode
  • pass the same codec to both sides of the managed LLM boundary
  • preserve the original provider response object when Relay does not change it
  • retain the existing untyped behavior for older Relay versions and unsupported provider protocols
  • keep non-interactive hermes --version probes offline while preserving update checks for interactive users
  • document the managed codec behavior

Why

The managed LLM integration already used protocol-specific execution names such as openai.chat_completions, but that name is telemetry identity rather than codec selection. As a result, Relay request interceptors could receive an unannotated request when Hermes ran through the native Python integration.

Explicitly passing Relay-owned codecs makes the managed boundary protocol-aware without adding a Hermes translation layer.

Relay also validates Hermes through a five-second, captured hermes --version probe. A source checkout previously performed a synchronous network update check for that non-interactive command, which could time out in a fresh isolated runtime home. Automated probes now remain local and deterministic; interactive version commands still report update availability.

Validation

  • 61 passed across the NeMo Relay plugin and version-command tests
  • Ruff checks pass for all modified Python files
  • a fresh isolated-home hermes --version probe completes in 0.53 seconds without a network update check
  • real Relay 0.6 codec smoke confirms a normalized request reaches the interceptor and the unchanged SDK response retains object identity
  • Fabric native-plugin and unmodified nemo-relay run --agent hermes smokes both complete with the same concrete dynamic plugin and each emit:
    • one canonical nemo_relay.llm.optimization contribution
    • a matching response optimization summary
    • 132 prompt/total tokens saved in the deterministic fixture

The local model has no pricing entry, so the smoke summaries are correctly partial for pricing while token evidence is complete.

@bbednarski9
bbednarski9 force-pushed the bbednarski/nemo-relay-managed-codecs branch from a1eb01f to 9dea284 Compare July 15, 2026 16:52
@bbednarski9 bbednarski9 changed the title fix(nemo-relay): pass provider codecs to managed LLM calls fix(nemo-relay): harden native and CLI integration Jul 15, 2026
@alt-glitch alt-glitch added type/bug Something isn't working comp/plugins Plugin system and bundled plugins comp/cli CLI entry point, hermes_cli/, setup wizard P3 Low — cosmetic, nice to have labels Jul 15, 2026
@teknium1

Copy link
Copy Markdown
Contributor

Thanks for the focused hardening work. Current main still supplies only the execution-surface string to nemo_relay.llm.execute at plugins/observability/nemo_relay/__init__.py:461-476, even though agent/conversation_loop.py:1366-1379 passes the resolved api_mode into the middleware. Current cmd_version also unconditionally enables update checks at hermes_cli/main.py:4434-4436, and the enabled path calls check_for_updates() at hermes_cli/main.py:4416-4421.

The codec selection, legacy fallback, response-identity coverage, and non-TTY version behavior directly address those current paths. GitHub reports the PR mergeable cleanly despite its older base.

Automated hermes-sweeper review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/cli CLI entry point, hermes_cli/, setup wizard comp/plugins Plugin system and bundled plugins P3 Low — cosmetic, nice to have sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants