fix(chat_completions): strip tool_name from messages for strict providers - #29594
Closed
rousegordon-ops wants to merge 186 commits into
Closed
fix(chat_completions): strip tool_name from messages for strict providers#29594rousegordon-ops wants to merge 186 commits into
rousegordon-ops wants to merge 186 commits into
Conversation
Railway rejects Dockerfile VOLUME instructions; persistence is provided by Railway Volumes mounted at /opt/data instead.
Bakes the messaging gateway as the default container command so Railway (and any other host) starts the right process without needing a custom start command in the dashboard.
Free, no API key, stdlib only. Subcommands: - 'at <place>' — geocodes via Nominatim, fetches weather - 'coords <lat> <lon>' — direct lookup for Telegram location pins
Hermes can autonomously create new skills (skill-authoring tools, agent-driven self-improvement). Without this, those changes live only on the persistent volume and get lost on volume reset / aren't visible to the next image build. The watcher (ported from GordonClaw's pattern) clones the fork into the volume on first boot, symlinks workspace skills/ to that checkout, and auto-commits + pushes any disk changes to origin/main within ~3 minutes. Conservative refresh logic preserves uncommitted/unpushed work across redeploys. Requires GITHUB_TOKEN env var (fine-grained PAT scoped to the fork with Contents: Write). When unset, watcher is silently disabled and Hermes runs as before.
Files: skills/.bundled_manifest
skills/.bundled_manifest is rewritten on every boot by skills_sync.py and was creating spurious 'auto:' commits from the watcher. The watcher log file is also pure runtime output. Both should never have been in source control.
Now that it's in .gitignore, also remove from the index so the watcher stops re-committing changes to it.
Sends OpenRouter balance + 24h spend (computed from snapshot delta) to TELEGRAM_HOME_CHANNEL every morning at 6 AM America/Los_Angeles. Pure Python, no LLM in the loop — deterministic format, zero ongoing token cost. v1 ships balance and total spend. Per-model breakdown (chat vs opencode), Tavily search count, and 5h peak request rate are deferred to v2 — those need either OpenRouter analytics access or a custom ledger hooked into Hermes' API call cycle. Daemon takes a baseline snapshot on startup (no Telegram send) when cost-state.json is missing, so the first scheduled report has a previous balance to diff against.
Files: skills/productivity/flights/
Files: skills/productivity/flights/SKILL.md
Symlink /opt/hermes/scripts and /opt/hermes/tools to the volume's git checkout (/opt/data/repo). New code pushed by the watcher (or by hand) shows up after a Railway redeploy without needing a full image rebuild — entrypoint refreshes the checkout, symlinks transparently expose new code to the gateway and daemons. The companion .github/workflows/auto-restart.yml fires on every push to main and routes by changed paths: - skill or docs only -> no-op (already live via the symlink) - dep / Dockerfile -> log and skip (manual rebuild required) - everything else -> railway redeploy (no rebuild) To complete the switch, disable Railway's 'auto-deploy on push' in the dashboard so this workflow is the sole trigger.
Two changes: 1. Dockerfile: chown /opt/hermes to hermes:hermes. Subdirs were already hermes-owned via COPY --chown, but the parent dir was root-owned, so the hermes user couldn't remove subdirs at runtime. 2. entrypoint.sh: the rm and symlink steps no longer crash the boot if they fail. They log a warning and fall through to using the image- baked copy. set -e was killing the entrypoint on permission denied, triggering Railway's restart loop.
…-agents/hermes-agent/SKILL.md skills/autonomous-ai-agents/hermes-agent/references/ Files: scripts/cost_report.py skills/autonomous-ai-agents/hermes-agent/SKILL.md skills/autonomous-ai-agents/hermes-agent/references/
Belt-and-suspenders defense against config.yaml drift. Yesterday a late volume mount caused the saved model setting to evaporate; the container fell back to anthropic/claude-opus-4.6 (the bundled default) and burned through ~$16 of Opus tokens before anyone noticed. This entrypoint addition runs 'hermes config set' on every boot for the two settings that have material safety implications. Idempotent (no-op when already correct) and guarded with || true so it can never crash the container. Override the values via Railway env vars when needed: HERMES_ENFORCED_MODEL HERMES_ENFORCED_APPROVALS
This fork only needs the auto-restart workflow. The upstream Hermes workflows (Tests, Supply Chain Audit, Nix Lockfile, Contributor Attribution Check, etc.) ran on every push + on upstream-mirrored PRs, generating failure emails that don't gate anything we care about and burning Action minutes.
Same pattern as scripts/ and tools/: replace /opt/hermes/gateway with a symlink to /opt/data/repo/gateway. Edits to gateway/run.py (e.g. an instrumentation hook for the request log) now actually persist on the volume, get pushed to GitHub by the watcher, and survive container restart instead of evaporating with the image filesystem on the next deploy. Was the missing piece for the request-log writer hook Hermes kept trying (and silently failing) to add — the edits were going to the image's gateway/run.py, never reaching the watched checkout, then getting wiped on the next restart.
Files: gateway/run.py
Railway's CLI uses different env var names depending on token type: RAILWAY_TOKEN is for project-deploy tokens, RAILWAY_API_TOKEN for account/workspace API tokens. The token in the GitHub secret is the account-scoped variety (only available scope on the dashboard), so the previous workflow runs failed with 'Invalid RAILWAY_TOKEN'. Setting both env vars from the same secret makes the workflow work regardless of which name the CLI ends up reading.
No functional change — testing the auto-restart workflow path now that the release-pin branch is configured and the RAILWAY_API_TOKEN env var fix is in place.
The CLI sees RAILWAY_TOKEN first, validates it as a project-deploy token (which the secret isn't), and exits with 'Invalid RAILWAY_TOKEN' without falling through to RAILWAY_API_TOKEN. Setting only the API token forces the right validation path.
Without these env vars the CLI fails with 'No linked project found' because there's no .railway directory in the runner. The IDs aren't secrets — they're resource identifiers, safe to commit.
…lf-rebuild/ skills/autonomous-ai-agents/hermes-agent/self-restart/ Files: skills/autonomous-ai-agents/hermes-agent/self-rebuild/ skills/autonomous-ai-agents/hermes-agent/self-restart/
…lf-rebuild/SKILL.md skills/autonomous-ai-agents/hermes-agent/self-restart/SKILL.md Files: skills/autonomous-ai-agents/hermes-agent/self-rebuild/SKILL.md skills/autonomous-ai-agents/hermes-agent/self-restart/SKILL.md
After a successful push, classify the changed paths into action buckets and send a Telegram message to TELEGRAM_HOME_CHANNEL when the change requires a container restart (Bucket 2) or image rebuild (Bucket 3). Bucket 1 changes (skills/docs/on-demand scripts) need no action and are silent. Watcher does NOT autonomously trigger Railway. The notification asks the operator to invoke self-restart or self-rebuild skill, which keeps human approval on every cost-incurring or downtime-incurring action. Bootstrap: the first deployment of this watcher requires a manual restart since the running watcher is the old version. Subsequent watcher updates self-notify.
MiniMax-M2.7 has a 204K context window and the new direct subscription doesn't throttle tokens, so the default 0.50 threshold (compress at ~102K) leaves half the window unused. 0.85 lets conversations grow to ~174K before compression kicks in, with 30K of headroom for the response itself. Env-overrideable via HERMES_ENFORCED_COMPRESSION_THRESHOLD.
…lf-rebuild/SKILL.md skills/autonomous-ai-agents/hermes-agent/self-restart/SKILL.md skills/autonomous-ai-agents/hermes-agent/references/railway-deployment.md Files: skills/autonomous-ai-agents/hermes-agent/self-rebuild/SKILL.md skills/autonomous-ai-agents/hermes-agent/self-restart/SKILL.md skills/autonomous-ai-agents/hermes-agent/references/railway-deployment.md
After every container restart or rebuild, post '👋 I'm back!' to TELEGRAM_HOME_CHANNEL. Best-effort — if Telegram is misconfigured or the API call fails, the boot continues normally.
Two changes: 1. Add TYPE_CHECKING import for RateLimitState. Line 1260 uses `Optional["RateLimitState"]` as a string forward reference but nothing imports the name, so ruff's F821 gate (which the watcher runs on every staged commit) was failing chronically and blocking any commit that touched run_agent.py. Hermes hit this trying to make the wording change below. TYPE_CHECKING-guarded import is the standard fix — costs nothing at runtime, satisfies ruff. 2. Shorten the fallback-activation status message. The "Primary model failed — switching to fallback:" phrasing is misleading when the primary hit a quota (it didn't "fail"), and the rate-limit emit above already says what's happening. Simplify to "Switching to X". Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…in skill Two independent additions: 1. functional-area-resolver — ported verbatim from garrytan/gbrain v0.32.3.0 (MIT). A/B-eval'd pattern for compressing RESOLVER.md / AGENTS.md from ~25KB to ~13KB while improving routing accuracy by +13-17pp across Opus 4.7, Sonnet 4.6, Haiku 4.5. Pure pattern guide; no code. Provenance note + path-reference adjustment added at top. 2. hermes-agent/brain — Hermes's own persistent memory at /opt/data/hermes-brain/. Deliberately minimal MVP: 3 directories (notes/, projects/, inbox/), markdown storage, no decision-tree resolver, no enrichment-on-signal, no hard search-before-answer rule. Inspired by gbrain's principles but a stripped-down subset to test whether wiki-as-context-saver hypothesis holds before committing to gbrain's full discipline. Strictly independent from skills/research/llm-wiki/ (Gordon's wiki at /opt/data/wiki/) — different storage, different audience, never cross-link. HTML rendering for human inspection is on-demand via the existing publish_html tool; markdown remains the storage format. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Observed 2026-05-15 that serviceInstanceRedeploy takes ~8 min BUILDING + ~1 min DEPLOYING, not the ~30s same-image restart the skill claimed. Either Railway changed behavior or the original claim was always wrong; treating it as rebuild-class until proven otherwise. Updates: - Description: explain observed behavior + treat-as-rebuild guidance - Body: add observed-duration section - Caution + Verification: change user heads-up from "~30s" to "~10 minutes (full rebuild)" - Confirmation: was "no confirmation required (cheap)" -> now require confirmation since rebuild-class is not a cheap unilateral op - Bump version 1.1.0 -> 1.2.0 GordonClaw's `gordonclaw-self-restart` skill is documented as a mirror; its docs likely have the same stale claim and may need the same fix. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Files: skills/creative/html-to-cloudflare/SKILL.md
Files: skills/creative/html-to-cloudflare/SKILL.md
Files: skills/research/llm-wiki/SKILL.md
Files: skills/research/llm-wiki/SKILL.md
The PKCE flow reused the code_verifier as the OAuth state parameter. Per RFC 6749 §10.12 and RFC 7636, these serve different purposes: state is an anti-CSRF token visible in the authorization URL; the code_verifier must remain secret for the token exchange. Generate an independent secrets.token_urlsafe(32) for state and validate it on callback to provide actual CSRF protection. Closes NousResearch#10693 (cherry picked from commit fcd9011)
Group the secrets import with time and webbrowser at the top of run_hermes_oauth_login_pure(), matching the existing pattern. Drop the _secrets alias — no name conflict in this scope. (cherry picked from commit 345821b)
…tion Two unit tests for run_hermes_oauth_login_pure(): 1. test_authorization_url_state_is_not_pkce_verifier — asserts state in the auth URL is independent from the PKCE code_verifier sent in the token exchange, and that the verifier never appears in the URL. 2. test_callback_state_mismatch_aborts — asserts the flow returns None (no token exchange) when the callback state does not match the value we generated. Negative control verified: reintroducing the b17e5c1 vulnerable pattern (state = verifier, no callback validation) makes both tests fail. Also adds AUTHOR_MAP entry for shaun0927 (contributor of the fix). (cherry picked from commit 72f94f4)
…t orphan _heartbeat_thread.start() was called before the try/finally block that contains _heartbeat_stop.set(). If _register_subagent() or any code between .start() and try: raised an exception, the finally block would never run — leaving the heartbeat thread as an orphan that continues calling _touch_activity() on the parent agent, incorrectly resetting gateway timeout counters. Move _heartbeat_thread.start() to be the first statement inside the try block so the finally block always reaches _heartbeat_stop.set() regardless of how the child run completes or fails. Root cause: heartbeat start outside try/finally scope Impact: orphan heartbeat thread incorrectly resets parent gateway timeouts (cherry picked from commit 2d7182f)
Pairs with the prior commit (start() now inside the try block). If
threading.Thread.start() itself raises (OS thread exhaustion under
heavy delegation fanout), the finally would call .join() on a
never-started thread, which raises RuntimeError("cannot join thread
before it is started") — trading one rare bug for another.
Thread.ident is None until start() succeeds, so gate the join on it.
(cherry picked from commit 6068363)
…NousResearch#26824) Subagent delegation hardcoded api_mode='chat_completions' for any delegation.base_url that didn't match three specific hostnames (chatgpt.com, api.anthropic.com, api.kimi.com/coding), and never read delegation.api_mode from config. Azure AI Foundry's https://foundry.services.ai.azure.com/anthropic endpoint fell through and got chat_completions, causing 404s on every delegate_task call. The main agent already handles this correctly via the shared _detect_api_mode_for_url() helper (anything ending in /anthropic → anthropic_messages); delegation reimplemented its own narrower check. Reuse the shared detector and honor an explicit delegation.api_mode when set so users can also force the transport on non-standard endpoints the URL heuristic can't classify. Fixes NousResearch#10213. Co-authored-by: HiddenPuppy <HiddenPuppy@users.noreply.github.com> (cherry picked from commit c445f48)
…text (NousResearch#26823) Adds _sanitize_tool_error() in model_tools and routes both error paths through it: registry.dispatch's try/except (the primary path for tool exceptions) and handle_function_call's outer except (defense in depth). Stripping targets structural framing tokens that the model itself can react to even though json.dumps already handles wire-layer escaping: XML role tags (tool_call, function_call, result, response, output, input, system, assistant, user), CDATA sections, and markdown code fences. Caps message body at 2000 chars and wraps with [TOOL_ERROR] prefix. Defense-in-depth: a tool exception carrying '<tool_call>...' won't break message framing (json escapes it), but the model still reads those tokens and they nudge it toward role-confusion framing. Ported from ironclaw#1639 (one piece of NousResearch#3838's three-feature scout). The truncated-tool-call (NousResearch#1632) and empty-response-recovery (NousResearch#1677, thoroughly (run_agent.py L8147/L12209/L13012 for truncation retry + length rewrite; L4500/L15090+ for empty-response scaffolding stripper, multi-stage nudge, fallback model activation). (cherry picked from commit 627f8a5)
… skills/github/github-pr-workflow/SKILL.md skills/github/github-pr-workflow/references/selective-cherry-picks-20260516.md Files: skills/creative/html-to-cloudflare/SKILL.md skills/github/github-pr-workflow/SKILL.md skills/github/github-pr-workflow/references/selective-cherry-picks-20260516.md
…ILL.md skills/autonomous-ai-agents/hermes-agent/references/gbrain-memory-path-separation.md Files: skills/autonomous-ai-agents/hermes-agent/SKILL.md skills/autonomous-ai-agents/hermes-agent/references/gbrain-memory-path-separation.md
…ILL.md skills/autonomous-ai-agents/hermes-agent/references/gordon-gbrain-path-layout.md Files: skills/autonomous-ai-agents/hermes-agent/SKILL.md skills/autonomous-ai-agents/hermes-agent/references/gordon-gbrain-path-layout.md
Files: skills/productivity/job-search/SKILL.md
Files: skills/productivity/job-search/SKILL.md
…/ skills/autonomous-ai-agents/archive-crawler/ skills/autonomous-ai-agents/article-enrichment/ skills/autonomous-ai-agents/book-mirror/ skills/autonomous-ai-agents/brain-pdf/ skills/autonomous-ai-agents/concept-synthesis/ (+3 more) Files: skills/autonomous-ai-agents/academic-verify/ skills/autonomous-ai-agents/archive-crawler/ skills/autonomous-ai-agents/article-enrichment/ skills/autonomous-ai-agents/book-mirror/ skills/autonomous-ai-agents/brain-pdf/ skills/autonomous-ai-agents/concept-synthesis/ skills/autonomous-ai-agents/perplexity-research/ skills/autonomous-ai-agents/strategic-reading/ skills/autonomous-ai-agents/voice-note-ingest/
…ferences/gordon-gbrain-path-layout.md uv.lock Files: skills/autonomous-ai-agents/hermes-agent/references/gordon-gbrain-path-layout.md uv.lock
When session_reset fires (idle/daily) or the gateway restarts, the agent's message history is empty — users experience this as "Hermes doesn't know what I was talking about." Read the N most recent user/assistant pairs from disk transcripts and append them to context_prompt, wrapped with a "consider only if relevant to the current message" instruction so they behave as reference material rather than the active conversation. Configurable via gateway.previous_exchanges_window (default 5; 0 to disable). Tool-call-only assistant turns are filtered out so the block shows just the user-facing text of each exchange. Per-message and per-file scan caps keep token cost and disk I/O bounded as transcripts accumulate. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Swap logger.debug to logger.info on the recent_exchanges hook so each inbound message logs whether the block was injected, was empty, or threw. Adds exc_info=True on the failure path so we see the traceback when something goes wrong inside collect_recent_exchanges. Pure observability change; no semantic difference. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ders The 'tool_name' key on role=tool messages is an internal Hermes field (stored in the messages.tool_name SQLite column for FTS indexing) that is not part of the OpenAI Chat Completions schema. Strict OpenAI-compatible providers — notably Moonshot AI (Kimi) — reject it with HTTP 400: Error from provider: Extra inputs are not permitted, field: 'messages[N].tool_name', value: 'execute_code' Add 'tool_name' to the sanitize block in ChatCompletionsTransport.convert_messages alongside the existing Codex Responses API fields (codex_reasoning_items, codex_message_items) so it is popped before the request is sent. Reproducer: hermes chat --model kimi-k2.6 > list the top 5 Hacker News stories -> assistant emits tool_call(execute_code) -> tool result message gets tool_name='execute_code' -> next turn's payload includes messages[N].tool_name -> 400 Permissive backends (MiniMax, OpenRouter on most routes) ignore the extra field and were masking the bug.
Author
|
Mis-targeted — meant to open against my fork (rousegordon-ops/hermes-agent), not upstream. Closing. |
13 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
16 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
258965663by @savanne-kham (upstream PR fix(chat_completions): strip tool_name from messages for strict providers #28958).tool_nameto the sanitize block inChatCompletionsTransport.convert_messagesso it's popped before requests are sent to strict OpenAI-compatible providers (Moonshot/Kimi, Fireworks).role=toolmessages carry our internaltool_namefield and strict providers reject the request with HTTP 400 (Extra inputs are not permitted).Why this one
Reviewed the upstream daily-update report (~2620 commits ahead). This is the lowest-risk, highest-value pull: 7-line transport-layer change, isolated from our gateway customizations, and protects any OpenAI-compatible routing we add later. The companion xAI OAuth base_url pin (
64a9a199b) was evaluated and skipped — our fork has diverged on that code path (test file deleted, 800+ lines of conflicts), and the attack surface it patches (tamperedXAI_BASE_URLleaking an OAuth bearer) doesn't apply here.Test plan
agent/transports/test_chat_completions.py(new test assertstool_nameis stripped)