fix(kanban): detect clean-exit workers reaped by init as protocol violations (#1096) - #3
Closed
mosaiq-systems wants to merge 188 commits into
Closed
fix(kanban): detect clean-exit workers reaped by init as protocol violations (#1096)#3mosaiq-systems wants to merge 188 commits into
mosaiq-systems wants to merge 188 commits into
Conversation
The new-project / add-folder dialog (PR NousResearch#49037) picked folders via the native Electron dialog (pickDefaultProjectDir), which only browses the LOCAL machine. On a remote gateway that picks a path that doesn't exist on the backend where sessions actually run. Route pickProjectFolder() through selectDesktopPaths({directories, multiple:false}) — the same remote-aware path the retired right-sidebar picker used: local mode opens the native directory dialog, remote mode browses the backend filesystem via the in-app RemoteFolderPicker. Seed it with the backend's default cwd on remote so it opens somewhere useful.
After the folder picker fix, an added remote folder was still half-usable: the desktop's git GUI (coding-rail status, worktree lanes, review pane, branch switch, file diff) all ran Electron-local git on the USER's machine, so against a remote-gateway repo they silently degraded to empty. Mirror the whole surface over the dashboard REST API so it acts on the BACKEND repo where sessions actually run: - hermes_cli/web_git.py: git/gh logic (status, worktrees, branches, review list/diff/stage/unstage/revert/commit/commit-context/push/ship-info/ create-pr, file-diff, worktree add/remove, branch switch) shelling to the system git, mirroring the Electron ops' shapes. - web_server.py: /api/git/* routes (same auth gate + _fs_path hardening as /api/fs, executor-offloaded, mutations -> 400). - apps/desktop desktop-git.ts: remote-aware facade exposing the same shape as window.hermesDesktop.git; coding-status / review / projects / model / desktop-fs route through desktopGit() so local stays Electron, remote hits /api/git/*. Tests: tests/hermes_cli/test_web_server_git.py (real repo: status counts, review classification, diff incl. untracked all-add, stage+commit roundtrip, worktree/branch lifecycle, commit-context, gh-absent ship-info, auth) and desktop-git.test.ts (local vs remote routing, envelope unwrap, POST bodies).
Collapse the two near-duplicate status parsers (_parse_status_v2 + _iter_status_entries) into a single _walk_entries generator feeding the rail, review list, and commit flow; share the staged predicate; hoist `import re`. Behavior unchanged.
writeProjectIdea used the local-only Electron writeTextFile, so on a remote gateway IDEA.md never landed on the backend (where the project folder lives). Route it through writeDesktopFileText (local Electron / POST /api/fs/write-text).
Second pass on the remote-project flow: the project dialog and git cockpit were remote-aware, but the composer's Add file/folder context picker still called the native Electron picker directly. Route it through selectDesktopPaths so remote sessions use the backend-aware picker instead of local disk paths; preserve local multi-select behavior and keep remote folder selection single because the in-app remote picker only supports one directory. Also use readDesktopFileDataUrl for image previews so an already-known backend image path can be read through /api/fs/read-data-url, and add focused coverage for backend file-diff routing plus the plain-folder git init/worktree path.
Keep the remote git mirror as a thin facade: route all GETs through gitGet, all mutations through gitPost, and keep consumers on desktopGit(). On the backend, route git paths through a single _git_path helper instead of repeating str(_fs_path(...)) in every endpoint. Behavior unchanged.
Let UI callers ask for folders/files without knowing remote-picker limits: selectDesktopPaths now normalizes remote directory selection to a single folder inside the facade. Project creation and composer context picking no longer branch on remote mode; they route through desktop-fs helpers just like git callers route through desktopGit(). Behavior unchanged except remote folder context now works through the same backend picker path.
Pin the desktop-to-gateway cwd handoff: createBackendSessionForSend must pass the current workspace cwd into session.create so the backend registers the session cwd before the agent/tools run.
…4254) Audited the last 3 releases (v2026.5.28..main) against the docs site and fixed code-vs-docs drift: - slash-commands: add /moa, /prompt, /pet, /hatch, /timestamps - cli-commands: add hermes pets / project / desktop / whatsapp-cloud + dashboard register; correct --insecure (now a deprecated no-op); add gateway migrate-legacy + enroll --wake-url + dashboard --skip-build - environment-variables: document the remaining ~48 env vars (SimpleX, Photon, Teams adapter, per-platform *_ALLOW_ALL_USERS, home-channel vars, IRC, Brave/Krea/Notion/Linear/Airtable/Tenor keys, QQ_SANDBOX) — full OPTIONAL_ENV_VARS (265) now covered - configuration: document tool_loop_guardrails, goals, prompt_caching, network, onboarding, dashboard config blocks - toolsets/tools-reference + tools.md: add coding/project toolsets and read_terminal/project_* tools; remove the stale messaging toolset and send_message agent tool (removed in NousResearch#47856); drop stale RL-training prose - messaging: new IRC channel page (adapter shipped without docs) + index row + sidebar + env vars - pets: document the /hatch AI generation pipeline + Nous/OpenRouter image backend - web-dashboard: document the bearer-token / TokenPrincipal service auth path - purge agent-callable send_message references across guides/features and the research-paper-writing skill (tool removed in NousResearch#47856) Verified: docusaurus build succeeds; all authored internal links resolve.
Route FS/git REST through the active profile, mount the remote folder picker at app root, keep the project dialog open while picking, show a first-run blank state, flip into grouped view on create, and constrain the picker scroll area so Select stays reachable.
The SSRF cluster (7a6fe9b, 48f5c42, 7ef04ae) sealed browser_snapshot, browser_vision, and _browser_eval against eval-navigated private pages, but browser_get_images bypasses _browser_eval and calls _run_browser_command("eval", ...) directly. An eval-driven navigation to a private address followed by browser_get_images would leak image src URLs and alt text from the private page. Add the same _eval_ssrf_guard_active + _current_page_private_url recheck before returning image data, matching the pattern established by the sibling guards. 5 new tests cover: block on private page, allow on public page, skip for local backend, skip when private URLs allowed, no guard needed on failed eval.
…ousResearch#40863) Removed/unauthorized Telegram users could inject prompt content before the per-user auth gate fired. The adapter ran `_should_process_message`, `_build_message_event`, and text/photo batching — and dispatched to the runner — before `_is_user_authorized()` (gateway/authz_mixin.py) rejected the sender. Unmentioned group chatter from a removed user was also persisted into the session transcript via `_observe_unmentioned_group_message`, leaking into the agent's observed context independent of dispatch. Add `_is_user_authorized_from_message()` as an intake prefilter that runs in `_handle_text_message`, `_handle_command`, `_handle_location_message`, and `_handle_media_message` BEFORE batching, event construction, and the unmentioned-group observe branch. It reuses the runner's `_is_user_authorized()` with a correctly-shaped SessionSource (group vs forum vs dm, real chat_id for TELEGRAM_GROUP_ALLOWED_* allowlists), falls back to env allowlists, and only rejects when an allowlist actually exists — unknown DMs with no allowlist still reach the pairing flow. Channel posts authorize via `sender_chat` identity when `from_user` is absent. Co-authored-by: liuhao1024 <sunsky.lau@gmail.com> Co-authored-by: Carlos Manuel Cejas <carlosmcejas@gmail.com>
CI git init defaults to master on some runners; compare branch to defaultBranch instead of pinning a branch name.
* fix(daytona): quote single-upload mkdir parent path
The single-file _daytona_upload() path shelled out 'mkdir -p {parent}'
with the remote parent interpolated unquoted, so shell metacharacters in
the path could break the command or inject arbitrary commands into the
sandbox. The bulk-upload, bulk-download, and delete paths were already
hardened with shlex-quoting helpers; this single-upload path was missed.
Route it through the existing quoted_mkdir_command() helper and add a
regression test covering a path with shell metacharacters.
Reported by @Gutslabs (NousResearch#3960); the original branch predated the
file_sync refactor, so the fix is re-applied to the current code path.
* docs(infographic): daytona quote-sync fix
…lder-picker-remote feat(desktop): remote-gateway-aware folder picker + git cockpit (status, review, worktrees)
…ch#52931) On Windows, uv pip install -e . can register hermes.exe in package metadata while the launcher never lands on disk. Detect missing [project.scripts] shims and reinstall entry points under the existing quarantine path in hermes update and install.ps1.
Add unit tests for missing-shim detection and repair trigger in _verify_console_scripts_installed.
…ing (NousResearch#54443) The curator's inactivity prune archived any non-pinned agent-created skill whose activity was older than archive_after_days (90d). A skill loaded only by a cron job had its usage bumped solely when the job fired, so paused jobs, infrequent (quarterly/annual) schedules, and far-future one-shots aged their skills out from under them — the next run then failed to load the now-archived skill. - cron/jobs.py: add referenced_skill_names() returning skills used by ANY job (incl. paused/disabled). - curator.apply_automatic_transitions(): skip cron-referenced skills like pinned; add a use=0 grace floor so a never-used skill is not marked stale/archived until it is at least stale_after_days old. - LLM review pass: candidate list marks cron=yes; prompt forbids pruning cron-referenced skills and never-used skills under 30 days. Tested E2E against a real cron job + real usage records and with 4 new unit tests.
…configured (NousResearch#3846) When config.yaml has `provider: auto` and a non-cloud `base_url` (e.g. Ollama at localhost:11434), requests were silently sent to https://api.anthropic.com whenever ANTHROPIC_API_KEY was present in the environment, ignoring the configured local endpoint and returning HTTP 401 / "credit balance too low". Root cause: resolve_provider("auto") scans env vars and returns "anthropic" when ANTHROPIC_API_KEY is set, before config.model.base_url is ever consulted. In resolve_runtime_provider(), before calling resolve_provider(), short-circuit to the OpenAI-compatible resolver when no explicit creds were passed, provider is "auto"/unset, and a non-cloud base_url is configured. Well-known cloud roots (openrouter.ai, anthropic.com, openai.com) are matched on HOST (not substring) so look-alike hosts can't evade the bypass and leak a cloud credential. Co-authored-by: Hermes Agent <hermes@nousresearch.com>
Bundled platform plugins (telegram, discord, feishu, teams, ...) were eagerly imported at plugin-discovery time on every `hermes` invocation, including plain `hermes chat` which never touches a gateway platform. Their modules import heavy platform SDKs at module level (lark_oapi, microsoft_teams, discord.py, slack_bolt, ...) — feishu alone pulled in lark_oapi (~2.6s), teams pulled microsoft_teams (~1.9s). Discovery now registers a cheap deferred loader per platform in the platform_registry; the adapter module is imported only when the gateway / cron / setup / send_message path actually asks for that platform. is_registered() and the iterate-all accessors stay correct (deferred counts as registered; plugin_entries()/all_entries() materialize all deferred loaders, since those paths genuinely need every adapter). Cold start: ~4.4s -> ~2.45s to banner. discover_and_load: 2.0s -> 0.3s (warm), and the heavy SDKs are no longer imported at all in CLI mode. Every shipped platform remains available out of the box — it just loads on first use.
…tion paths A config left with `provider: anthropic` but a leftover `base_url: https://openrouter.ai/api/v1` (e.g. after a provider switch) would route Anthropic OAuth/setup-token traffic to OpenRouter and 404. Add `_anthropic_base_url_override_ok()` and gate the three native-Anthropic resolution branches (pool, explicit, native) on it. The guard honors a configured `model.base_url` only when it plausibly speaks the Anthropic Messages protocol — official `*.anthropic.com` / `*.claude.com` hosts, Azure Foundry endpoints, and `/anthropic`-suffixed or Kimi `/coding` proxies — and falls back to `https://api.anthropic.com` otherwise. Aggregator URLs like openrouter.ai / api.openai.com are treated as stale. Reconstructed from @clovericbot's PR NousResearch#3661 onto current main: the original patched one branch with an anthropic-only allow-list, which would have broken Azure-via-anthropic; widened to all three sites and made Azure/proxy-safe.
…nsole-launcher-repair fix(windows): repair missing console script launchers
) yuanbao_media.download_url() fetched model-supplied (outbound) and inbound image/file URLs server-side via httpx with follow_redirects=True and no SSRF check. A model response containing <img src="http://169.254.169.254/..."> routed through ImageUrlHandler -> download_url and would fetch cloud-metadata endpoints; same for inbound media. Add an is_safe_url() pre-flight plus an async redirect event-hook that re-validates every 30x target, matching the cache_image_from_url() guard in gateway/platforms/base.py. The other gateway adapters already guard their URL-fetch paths; this was the remaining unguarded one.
…er (NousResearch#54486) The startup config/manifest reads used PyYAML's pure-Python SafeLoader, which is ~8x slower than the libyaml-backed CSafeLoader C extension. config.yaml is parsed several times during launch (cli config, raw config, early interface/redaction bridge, logging config) and every plugin manifest is parsed once — all on the slow path. Add utils.fast_safe_load (CSafeLoader-preferring, pure-Python fallback, true drop-in for safe_load) and route the hot startup parse sites through it: hermes_cli/config.py (config + manifest reads), hermes_cli/plugins.py (manifest parse), env_loader, cli.load_cli_config, hermes_logging, and the two pre-config early YAML bridges in main.py. Behavior is identical (same restricted safe tag set); only speed changes. safe_load calls on the startup path drop from ~79 to ~0, cutting the YAML parse cost from ~0.9s to ~0.15s under profiling. Adds tests/test_fast_safe_load.py asserting equivalence with safe_load across input shapes, empty-doc falsiness, C-loader preference, and that python/object tags are still rejected (safe, not full loader).
…obes The NousResearch#54236/NousResearch#54417 backend git/gh sweep routed git_probe, the repo-file picker, coding_context, context_references, copilot_auth, and the gateway process scans through CREATE_NO_WINDOW, but two sibling spawn legs that also run inside the console-less desktop/gateway backend were missed: - tools/checkpoint_manager.py `_run_git` (and the one-shot `git init --bare` in `_init_store`) — when checkpoints are enabled, every file-mutating turn fires multiple bare `git` calls (status, add, write-tree/commit-tree, update-ref). Spawned from a parent with no console (Electron spawns the backend with windowsHide → CREATE_NO_WINDOW), each one allocates its own conhost window → a flurry of terminal popups. - tools/skills_hub.py `GitHubAuth._try_gh_cli` — `gh auth token`, the same bug class as the already-fixed copilot_auth gh probe. Route both through `windows_hide_flags()` (no-op on POSIX), matching the established per-site pattern. Tests added to tests/test_windows_subprocess_no_window_flags.py.
usePetRoam re-measures ledges from the live DOM each beat and walks/hops/falls between them, driving DOM position imperatively (no per-frame re-render).
Tag both bars with data-slots; the roam loop stands on the status bar's top edge (not over it) and treats the profile rail as a climbable ledge.
Drop the CSS lens overlay (blend modes, noise, inversion) and backdrop-blur from the ops dashboard so compositing no longer competes with xterm on /chat. Use flat theme backgrounds and direct Nous Blue palette colors instead of FG-inversion authoring. Co-authored-by: Cursor <cursoragent@cursor.com>
Make Nous Blue terminal text readable without the inversion layer, re-mount the backdrop plugin slot, and drop unused backdrop CSS vars from theme apply. Co-authored-by: Cursor <cursoragent@cursor.com>
Use momentum easing for sidebar transitions, switch sidebar typography to sans-serif, replace the profile native select with the DS Select, and stop clipping the Models page Use-as dropdown inside model cards. Co-authored-by: Cursor <cursoragent@cursor.com>
Prompt before restarting from the sidebar system menu, and replace native checkboxes on the System page with the design-system Checkbox component. Co-authored-by: Cursor <cursoragent@cursor.com>
Match the Restart Gateway flow with a confirm dialog that fetches cached update metadata so users see commit-behind context before applying. Co-authored-by: Cursor <cursoragent@cursor.com>
…urface Walk speed is derived from the sprite's animation loop + on-screen size (one body-width per loop) instead of a fixed px/s, so it steps rather than glides; the pet also sinks a few px so its feet meet the surface instead of hovering.
When a full-screen route overlay (settings/profiles/cron/agents/command-center) is up, the pet's walkable surface swaps to a single ledge at the overlay card's bottom edge — derived from OverlayView's shared inset, not measured — so it patrols there; closing the overlay restores the normal surfaces and it drops back down.
feat(desktop): roaming pet (opt-in)
…locked bots) (NousResearch#55115) * fix(gateway): skip confirmed-dead delivery targets (deleted groups, blocked bots) A deleted Telegram group, kicked/blocked bot, or deactivated user keeps throwing Forbidden/not_found on every cron tick and fan-out delivery. Each retry burns a send against the platform's flood-control envelope and spams the logs, making the whole session feel broken even when the model call completed. Add a small persistent DeadTargetRegistry (per-profile JSON under HERMES_HOME) that records a target the moment a send reports a whole-chat death (forbidden / chat-level not_found), and have DeliveryRouter.deliver() short-circuit it on subsequent attempts. Self-healing: any successful send clears the flag, so a user re-adding the bot recovers with no manual cleanup. Thread/topic-level not_found is NOT recorded (adapters already self-heal that by retrying without reply_to). Transient/timeout errors are never marked dead. * infographic: dead delivery target skipping
…to-speak-replies feat(desktop): read replies aloud (auto-TTS) composer toggle
The migration's call-site sweep keyed on the literal self._session_db. spelling and missed calls bound to a local first (db = getattr(self, '_session_db', None); db.method(...)). Convert the three in async contexts: get_telegram_topic_binding in the topic-rename coroutine, and the two update_session_model sites on the model-switch path.
… loop The topic-mode helpers (_telegram_topic_mode_enabled, _recover_telegram_topic_thread_id, _record/_sync_telegram_topic_binding, _is_telegram_topic_lane/_root_lobby, _normalize_source_for_session_key, _telegram_topic_new_header, _schedule_telegram_topic_title_rename, and the base.py _apply_topic_recovery hook) each run a synchronous SessionDB read or write. They reach the event loop through async handlers, so a contended state.db froze the loop the same way the handoff watcher did. These helpers already run off-loop in the run_sync thread-pool closure, so they are proven thread-safe there. Rather than colour them async, loop-side callers now invoke them via asyncio.to_thread(...); the executor callers are unchanged. Inside the helpers the SessionDB handle is unwrapped to the sync door (getattr(db, '_db', db)) since they always run on a worker thread, and AIAgent construction + query_session_listing are handed the sync SessionDB directly. base.py wraps its single _apply_topic_recovery call in to_thread. The guard is now alias-aware (catches db = getattr(self, '_session_db', None); db.method(...)) and enforces the offload contract: the offloaded sync helpers may never be called bare on the loop. Sibling test fixtures wrap their injected SessionDB in AsyncSessionDB to match how the gateway holds it.
…ect_uri (NousResearch#55099) The self-hosted OIDC dashboard login rejected any http:// redirect_uri whose host was not localhost/127.0.0.1, surfacing "redirect_uri may only use http:// for localhost/127.0.0.1" before reaching the IDP. This broke self-hosted dashboards reached over plain HTTP (including LAN IPs, internal hostnames, and reverse proxies that terminate TLS upstream). NousResearch#38827 already dropped this check from the nous provider, but the generic self-hosted provider copied the old localhost-only branch and reintroduced the bug for HERMES_DASHBOARD_OIDC_ISSUER setups. The IDP's own allowlist is authoritative on which redirect_uris are permitted; this client-side _validate_redirect_uri is only a fast-fail for obvious operator error and should not second-guess valid http:// deployments. Fix: drop the localhost-only branch on the http scheme. Validation now enforces only that the scheme is http(s) and the path ends with /auth/callback. Updated the docstring to explain the relaxed contract, and added test_allows_http_with_arbitrary_host covering an internal hostname and a LAN IP alongside the existing localhost case.
…ad/write) (NousResearch#55289) Gateway half of relay-platform-parity Phase 2.5 (D-Q2.5). The relay wire's platform-neutral scope discriminator is renamed guild_id → scope_id; this is the hermes-agent side of the cross-repo wire-compatible migration. - SessionSource: scope_id is canonical; guild_id kept as @deprecated alias. __post_init__ mirrors the two so all existing SessionSource(guild_id=...) constructors across native adapters keep working unchanged. to_dict dual-WRITES scope_id+guild_id; from_dict dual-READS scope_id ?? guild_id. - relay/adapter.py: capture + outbound metadata dual-read/write scope_id. - relay/ws_transport.py: _frame_to_event dual-reads scope_id ?? guild_id. - docs/relay-connector-contract.md: document scope_id (canonical) + guild_id (deprecated alias) in the §3 SessionSource field table (conformance test). 250 relay+session+contract tests green. Solo lane (relay).
…lations (NousResearch#1096) When a worker exits cleanly (rc=0) without calling kanban_complete, init or a subreaper can reap the zombie before the dispatcher's reap_worker_zombies() loop captures the exit status via waitpid(). This causes _classify_worker_exit() to return "unknown" instead of "clean_exit", and the task is treated as a regular crash — retried until the circuit breaker trips, burning dispatch capacity each cycle. Add a lightweight Popen-handle registry (_active_worker_popens) that stores the subprocess.Popen object at spawn time. When the reap registry has no entry for a dead PID, _classify_worker_exit() now polls the stored Popen handle to recover the exit code even after init has reaped the zombie. Clean exits are correctly classified as protocol violations and auto-blocked on the first occurrence. Changes: - Add _active_worker_popens dict + _ACTIVE_WORKER_POPEN_TTL_SECONDS - Add _register_worker_popen() called from _default_spawn() - Add _poll_worker_handle() to recover exit codes from Popen handles - Add _cleanup_stale_popen_handles() for per-tick garbage collection - Modify _classify_worker_exit() to poll handles before returning unknown - Wire _cleanup_stale_popen_handles() into _dispatch_once_locked() Task: t_f56bf72a
Owner
Author
|
Superseded by PR #4 (clean branch from main with only the kanban_db.py fix). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix for zombie protocol violation (NousResearch#1096)
When a worker exits cleanly (rc=0) without calling kanban_complete,
init or a subreaper can reap the zombie before the dispatcher's
reap_worker_zombies() loop captures the exit status via waitpid().
This causes _classify_worker_exit() to return "unknown" instead of
"clean_exit", and the task is treated as a regular crash — retried
until the circuit breaker trips, burning dispatch capacity each cycle.
Changes
_active_worker_popensdict +_ACTIVE_WORKER_POPEN_TTL_SECONDS_register_worker_popen()called from_default_spawn()_poll_worker_handle()to recover exit codes from Popen handles_cleanup_stale_popen_handles()for per-tick garbage collection_classify_worker_exit()to poll handles before returning unknown_cleanup_stale_popen_handles()intodispatch_once()Verification
test_detect_crashed_workers_increments_counterfails due to pre-existing test guard issue, unrelated)Task: t_f56bf72a