[codex] fix dflash malformed final recovery - #48
Closed
OmarB97 wants to merge 31 commits into
Closed
Conversation
…search#35729) The all/* wildcard expands to every registered toolset, but a handful of tools have an additional check_fn gate on top of toolset membership and are intentionally NOT turned on by all/* alone: - Capability-gated tools (browser, computer_use, code_execution, Feishu, Home Assistant, cronjob) require their backend/credential prerequisite. - The kanban toolset is workflow-gated and deliberately opt-in. Kanban tools mutate shared board state, so they stay off by default even under all/* — you must list 'kanban' by name (or be a dispatcher-spawned worker with HERMES_KANBAN_TASK set). This was the expectations gap behind NousResearch#35581 — the docs previously said all/* expands to 'every registered toolset' without noting the carve-out. Closes NousResearch#35581.
…ousResearch#35719) SSH sessions hard-failed voice mode on the presence of SSH_* env vars alone, even when a PulseAudio/PipeWire server is running on the host and audio works (ffplay/aplay/pw-play -> pulseaudio). Probe the default sound-server sockets (PULSE_SERVER unix path, PULSE_RUNTIME_PATH/native, $XDG_RUNTIME_DIR/{pulse/native,pipewire-0}) and actually connect() so a stale socket doesn't count; downgrade the SSH branch to a notice when audio is reachable. Mirrors the existing Docker/WSL forwarding handling. Fixes NousResearch#35622
…Research#35732) Resize vision tool-result images down to a 4 MB embed cap at load time, not just at the 20 MB hard ceiling. A 5-20 MB image previously sailed through the native fast path and got baked into conversation history, where Anthropic's 5 MB per-image base64 limit rejected every subsequent turn with a 400 — and because history is immutable, retries could never clear it, permanently wedging the session. Also harden the reactive shrink-recovery: it now returns False (don't retry) when any oversized image part can't be brought under target, so the single retry isn't burned re-sending a payload that will fail identically. Previously it returned True after shrinking *any* part, even when the actual oversized culprit survived.
…d providers (NousResearch#35718) DeepSeek / Baidu Qianfan stream tool-call arguments in cumulative mode: each chunk resends the full arguments-so-far instead of the new fragment. The stream accumulator blindly concatenated arg deltas with +=, turning that into '{...}{...}{...}', which failed json.loads and got nuked to '{}' — a silently corrupted tool call (NousResearch#35592). Worse on multi-param tools (search_files, session_search, memory replace) because longer args take more chunks, giving more resend opportunities. - Per-slot cumulative latch in the stream accumulator: a delta that is a strict superset of the accumulated buffer marks the slot cumulative and replaces (not appends); exact duplicates are dropped only after latching. Incremental fragments are untouched (default += path). - Backstop _collapse_repeated_json_arguments() in the repair pipeline collapses pure identical-resend buffers (K exact repeats of a valid-JSON unit) for providers that resend the complete object from chunk 1. Only reached after json.loads already failed, so compliant single objects are never touched. Not a gateway or DeepSeek-model bug — any OpenAI-wire provider in cumulative streaming mode is affected.
…arch#35756) Lower the model_catalog disk-cache TTL from 24h to 1h so freshly published model-catalog.json deploys reach the picker within an hour instead of up to a day. The picker now refetches on the next `hermes model` / `/model` once the cache is older than 1h; younger than 1h still serves the cache (no network hit), and network failures still fall back to the stale copy. - DEFAULT_TTL_HOURS 24 -> 1 (model_catalog.py) - DEFAULT_CONFIG model_catalog.ttl_hours 24 -> 1, _config_version 24 -> 25 - migration v24->25 rewrites a stale ttl_hours:24 to 1, preserving any custom value the user set E2E: verified >1h refetches / <1h skips, and migration rewrites 24->1 while preserving a custom 6.
…ounded growth _message_text_cache was a plain dict with no size limit. Every unique message_id whose text was fetched (for reply-context lookups) stayed in memory permanently, causing unbounded growth in long-running deployments with active group chats. Replace with an OrderedDict and evict the least-recently-used entry whenever the cache exceeds _FEISHU_MESSAGE_TEXT_CACHE_SIZE (512). Cache hits call move_to_end() to refresh LRU order. Mirrors the identical pattern already used by _pending_processing_reactions in the same class.
…ded growth The _guid_cache dict grows without bound as new contacts/groups are resolved. In a long-running gateway instance with many unique targets this becomes a slow memory leak. Replace the plain dict with an OrderedDict capped at 500 entries. When the cap is exceeded the oldest (least-recently-used) entries are evicted.
…nnect poll
PAIN BEFORE:
Inside _handle_auth_error_and_retry() (a sync function that runs on the MCP
event loop thread), there was a blocking polling loop:
while time.monotonic() < deadline:
if srv.session is not None and srv._ready.is_set():
break
time.sleep(0.25) # BLOCKS THE ENTIRE EVENT LOOP
Since _handle_auth_error_and_retry is invoked from tool handlers that run ON
the MCP event loop, time.sleep(0.25) blocked ALL concurrent MCP operations
(including other tools, keepalive heartbeats, OAuth refreshes) for 250ms per
iteration. With a 15-second deadline, worst case = 60 * 250ms = 15 seconds
of fully blocked concurrency.
WHAT WAS FIXED:
Extracted the blocking poll into an async helper _await_ready() that uses
asyncio.sleep(0.25) (non-blocking), and runs it via _run_on_mcp_loop().
_run_on_mcp_loop() properly awaits the coroutine on the event loop without
blocking the caller's thread. Added exception handling around the poll so
stuck reconnects still fall through to the error path.
The sync _handle_auth_error_and_retry now:
1. Fires reconnect signal (threadsafe)
2. Calls _run_on_mcp_loop(_await_ready(), timeout=15) — non-blocking
3. Returns; the event loop handles the polling
File: tools/mcp_tool.py
Lines: _handle_auth_error_and_retry() (~1886-1920)
Found by: exhaustive multi-pass audit (10 strategies, 1901 files, 913K lines)
…her recovery N43 — Silent plugin/bundle errors: - Plugin command dispatch: logger.debug() -> logger.warning() - Bundle dispatch: logger.debug() -> logger.warning() Plugin/auth failures are no longer invisible to operators. N42 — O(n^2) pending_watchers recovery: - Both recovery loops (startup + per-message) used while+pop(0) which is O(n) per pop - Replaced with enumerate() over the list + periodic asyncio.sleep(0) yield points - Clears the list after iteration instead of per-pop - Batch size of 100 balances throughput vs event-loop responsiveness
…align test fixtures + AUTHOR_MAP Self-review follow-up on top of the salvaged perf fixes: - gateway/run.py (both watcher-drain sites): the salvaged O(n^2) fix (NousResearch#32708) replaced `while pending_watchers: pop(0)` with iterate-then- `watchers.clear()`, but `watchers` aliased the registry's live list. A watcher appended by a concurrent session during the `await asyncio.sleep(0)` yield would be cleared without ever being scheduled. Detach the batch atomically (`pending_watchers = []`) before iterating. - gateway/platforms/bluebubbles.py: normalize the salvaged _guid_cache LRU (NousResearch#30523) to match feishu/codebase precedent — module-level `_GUID_CACHE_SIZE` constant, `while len > cap`, and drop the redundant post-insert `move_to_end` (a fresh insert is already most-recent). - gateway/platforms/feishu.py: drop the same redundant post-insert `move_to_end` from the salvaged _message_text_cache LRU (NousResearch#23706). - scripts/release.py: add AUTHOR_MAP entries for the salvaged commits' authors (amathxbt NousResearch#22155, ErnestHysa NousResearch#32636/NousResearch#32708) so the contributor audit passes when these commits land on main. - tests/tools/test_tool_output_limits.py: autouse fixture resets the new module-level limits cache between tests. - tests/gateway/test_feishu.py: hand-built adapter fixture seeded _message_text_cache as a plain dict; it's now an OrderedDict, so the fixture type had to match.
…ch#35710) * feat(kanban): goal_mode cards run workers in a /goal loop A goal_mode card wraps its dispatched worker in the Ralph-style goal loop behind /goal: after each turn an auxiliary judge checks the worker's response against the card title+body, and if not done the worker keeps going in the SAME session until the judge agrees, the worker terminates the task itself, or the turn budget runs out (which blocks the card for human review — never a silent exit). - kanban_db: goal_mode + goal_max_turns columns (additive migration), Task fields, create_task params, INSERT wiring, created-event payload. - kanban_tools: goal_mode/goal_max_turns on the kanban_create tool so orchestrators can opt cards in when fanning out. - kanban CLI: --goal / --goal-max-turns on 'kanban create'. - dashboard API: goal_mode/goal_max_turns on the create endpoint (auto-surfaced back via asdict). - _default_spawn: sets HERMES_KANBAN_GOAL_MODE / _GOAL_MAX_TURNS only when the card opts in. - goals.run_kanban_goal_loop: standalone, callback-injected loop engine (no SessionDB persistence; ephemeral worker). cli.py quiet path calls it after the worker's first turn when the env vars are set. - Docs: orchestrator skill + kanban feature page. Tests: DB roundtrip + legacy migration, spawn env gating, and the loop's continuation/completion/budget-block/finalize-nudge branches. E2E run against a real kanban DB confirms a budget-exhausted goal worker lands in a sticky blocked state. * feat(kanban/dashboard): goal-mode toggle in the create form Wires the goal_mode card setting into the dashboard UI (the plugin's hand-written IIFE bundle, no build step): - InlineCreate: 'goal mode' checkbox after the skills field; checking it reveals an optional 'max turns' number input. Both reset on submit and only post goal_mode/goal_max_turns when enabled. - TaskDrawer: a 'Goal mode: on (max N turns)' MetaRow so a card's goal-mode setting is visible after creation (auto-fed by asdict via the existing _task_dict). Live-tested through the running dashboard with a browser: created a goal-mode card with max-turns=8, confirmed it persisted to the kanban DB (goal_mode=1, goal_max_turns=8) and rendered back in the drawer as 'on (max 8 turns)'. No JS console errors.
The setup wizard's provider/model pickers (curses_radiolist via prompt_choice) bailed to the numbered "Select [1-N]" fallback the moment a user pressed up or down. Root cause: even with keypad(True) — which curses.wrapper sets — many terminals/terminfo entries deliver cursor keys to getch() as raw CSI/SS3 byte sequences (e.g. 27, 91, 66 for arrow-down) rather than the translated curses.KEY_DOWN. The menus matched only curses.KEY_UP/KEY_DOWN and treated the leading 27 (ESC) as cancel, so navigation dropped into the text fallback and the trailing bytes leaked into the next input(). Add a shared read_menu_key() helper that decodes CSI/SS3 escape sequences into normalized NAV_* actions (only a lone ESC, with no continuation byte within a short timeout, still cancels) and consumes the tail of unhandled sequences so stray bytes can't corrupt later input(). Route all three curses menus (checklist, radiolist, single_select) through it. Add regression tests covering raw CSI/SS3 arrows, translated KEY_* constants, vim keys, lone-ESC cancel, and full consumption of unhandled sequences (Delete/Home/End).
…rrow-key-decode fix(cli): decode raw arrow-key escape sequences in curses menus
…o curses The setup provider->model sub-menu (and three sibling pickers) used simple_term_menu.TerminalMenu, whose ESC and arrow-key handling was unreliable across terminals — notably ESC failed to back out of the model selection list on terminals that emit raw escape sequences (e.g. Ghostty). The codebase already notes simple_term_menu 'conflicts with /dev/tty' and causes 'ghost-duplication rendering', and a prior attempt to migrate these (closed PR) confirmed the same root cause. Route all four single-select pickers through the shared, already-hardened curses_radiolist (which decodes raw CSI/SS3 escape sequences and handles ESC consistently, fixed in NousResearch#35776): - auth.py _prompt_model_selection — model picker; the pricing column header and the unavailable-models block are passed as the radiolist description so they survive the curses screen clear. ESC now cancels. - main.py _prompt_reasoning_effort_selection — reasoning-effort picker. - main.py _model_flow_named_custom — named custom-provider model picker. - main.py _remove_custom_provider — provider-removal picker. simple_term_menu is no longer imported anywhere (only stale comments referenced it; one in setup.py is corrected). The numbered-input fallbacks are unchanged and still trigger on curses errors / non-TTY. Tests: updated test_terminal_menu_fallbacks / test_reasoning_effort_menu / test_custom_provider_model_switch / test_model_provider_persistence to drive the fallback via curses_radiolist errors instead of breaking simple_term_menu. New test_setup_menu_curses_migration.py asserts each picker routes through curses_radiolist, ESC cancels, and the pricing header is preserved. Net -147/+183 (mostly the new test file; production code shrinks by removing TerminalMenu boilerplate).
The three curses menus (curses_checklist / curses_radiolist / curses_single_select) each hand-rolled an identical event loop: cursor hide + color-pair init, the per-frame clear/getmaxyx/refresh cycle, scroll-offset math, row iteration, the read_menu_key dispatch with NAV_UP/NAV_DOWN cursor wrap, flush_stdin, and the KeyboardInterrupt/curses-unavailable fallback. Terminal-behavior changes (e.g. Ghostty raw-escape handling, scroll tweaks, a new key) had to be made in three places. Extract that boilerplate into one _run_curses_menu driver. Each public menu now supplies small callbacks for the parts that genuinely differ: draw_header (returns the item-list start row), draw_row (checkbox vs radio vs bare prefix), an on_action reducer (toggle-set vs return-cursor vs return-None + the single_select cancel-row guard), an optional draw_footer (the checklist status bar), reserve_bottom, and the numbered fallback. Behavior is passed as functions; the loop is the only stateful piece — so future terminal/Ghostty work is a one-place edit. Duplicated event-loop primitives drop 3 -> 1 (stdscr.clear, read_menu_key dispatch, scroll math). Verified byte-identical: a render harness records every addnstr(y, x, clamped-text, attr) call across frames plus the return value for 6 cases (checklist, checklist+status, radiolist, radiolist+description, single_select, single_select ESC-cancel); output diffs clean against origin/main. Non-TTY returns the cancel value directly (not the input()-based numbered fallback), matching the old per-menu guard. 150 menu/setup/browse/plugins tests pass.
…ousResearch#35792) * feat(tools): always show Nous Tool Gateway backends, login on select The Nous-managed Tool Gateway rows in `hermes tools` (Firecrawl, OpenAI TTS, Browser Use, FAL image/video) were hidden unless the user was already logged into Nous Portal with paid access. Now they are always listed. Selecting one runs an inline Nous Portal device-code OAuth + entitlement check — auth only, no inference-provider switch and no bulk 'enable all tools' prompt (that stays in `hermes model`). The row only activates the gateway once paid access is confirmed. - _visible_providers: stop hiding managed_nous_feature rows (incl. those also flagged requires_nous_auth); pure pre-auth UX rows still gate on login - nous_subscription.ensure_nous_portal_access(): auth + entitlement gate that preserves the user's active inference provider - _configure_provider / _reconfigure_provider: run the inline gate for managed backends; write config only when entitled - picker marker: 'via Nous Portal (login on select)' for logged-out users - _hidden_nous_gateway_message: now a no-op (rows are never hidden) * docs: hermes tools is a first-class Tool Gateway entry point The Tool Gateway docs framed `hermes setup --portal` / `hermes model` as the activation path and only mentioned `hermes tools` for mixing in your own keys. With the inline-login change, picking a Nous-managed backend in `hermes tools` is a complete path on its own — it logs you into Nous Portal on select if needed, without switching your inference provider or prompting to enable every other tool. - tool-gateway.md: Get started now lists three peer entry points; new paragraph explaining login-on-select and the no-prompt fast path when OAuth is already active - nous-portal.md + run-hermes-with-nous-portal.md: note that managed rows appear logged-out and trigger inline login on select
…search#35858) The status bar read context_compressor.last_prompt_tokens directly with an 'or 0' guard that only catches 0/None. Right after a compression the compressor parks last_prompt_tokens at the -1 sentinel (awaiting_real_usage_after_compression) until the next API call reports real usage. -1 is truthy, so it sailed through and rendered as '-1/200K' and '-1%' for that one transitional turn. Clamp negative token/context-length values to 0 in the status-bar snapshot so the gap reads as empty context until real usage arrives.
…ng path (NousResearch#35718) (NousResearch#35860) PR NousResearch#35718 added a per-slot "cumulative-resend" latch to the universal streaming tool-call accumulator to fix DeepSeek / Baidu Qianfan (NousResearch#35592). The latch fires when a delta is a strict superset of the accumulated buffer (len(_new) > len(_prev) and _new.startswith(_prev)) and then REPLACES the buffer instead of appending. That superset test is not an unambiguous cumulative signature. A normal incremental stream can emit a single fragment that restates an already- accumulated prefix — trivially common in large code-patch arguments with repeated lines / indentation — which trips the latch and clobbers the accumulated buffer, corrupting the tool call. Observed in the wild on Anthropic Opus (the primary model) building a large patch: corrupted / short arguments → finish_reason='length' dead-end → session killed. A guessing heuristic that can silently clobber a tool-call buffer has no place on the path every provider and model shares. Reverting restores the known-good plain `+=` accumulator. The NousResearch#35592 narrow provider bug should be re-addressed provider-gated so it is structurally impossible to touch Anthropic / OpenAI incremental streams, rather than via a heuristic on the shared path. Reverts ca03486.
…tes the latest turn
Extended-thinking Claude models (4.6+, e.g. Opus 4.8) emit a signed `thinking`
block on assistant turns that also carry parallel `tool_use` blocks. Anthropic
signs that block against the full, original turn content.
When a parallel tool batch is interrupted before every `tool_result` returns,
`_strip_orphaned_tool_blocks` removes the unanswered `tool_use` on replay — which
mutates the turn. The latest-assistant branch of `_manage_thinking_signatures`
then replays the now-stale signed thinking block verbatim, and Anthropic rejects
the request with a non-retryable HTTP 400:
messages.N.content.M: `thinking` or `redacted_thinking` blocks in the latest
assistant message cannot be modified. These blocks must remain as they were
in the original response.
Because the poisoned turn is rebuilt from the persisted store every turn, the
gateway crash-loops with no self-recovery (a soft session reset does not clear
it). The drifting content index in the error is the changing count of stripped
`tool_use` blocks across rebuilds.
Fix: when orphan-stripping removes a `tool_use` from a turn that also holds a
thinking/redacted_thinking block, flag the turn. `_manage_thinking_signatures`
then demotes every thinking block on that latest turn to a plain text block
(preserving the reasoning text) instead of replaying a signature that can no
longer validate. An intact turn is unaffected — its signed thinking is still
replayed verbatim. The internal flag is stripped before the payload is sent.
Adds two regression tests:
- demotion when an orphaned parallel tool_use is stripped
- control: signed thinking preserved verbatim when nothing is stripped
… lifecycle breadcrumbs) (NousResearch#35893) * fix(tui): persist gateway lifecycle breadcrumbs to crash log A backend SIGTERM (`=== SIGTERM received ===` in tui_gateway_crash.log) is always a parent action — `gw.kill()` (graceful-exit on a signal to Node, or an explicit /quit) or `start()` replacing a live child. NousResearch#31051 added parent-side lifecycle breadcrumbs but left them in an in-memory CircularBuffer that dies with the process, so SIGTERM crash reports arrive with no parent context and no way to tell a signal-driven kill from a memory-critical `process.exit(137)` (which closes the child's stdin → clean EOF, not SIGTERM). Persist the death-explaining breadcrumbs (spawn / transport-exit / child-exit / replace-live-child / kill-reason / startup-timeout) plus the graceful-exit signal name and the memory-critical exit into the same crash log the Python side writes, so they interleave by timestamp next to the child's panic entry — making these recurring reports diagnosable. Gated off under VITEST so unit tests stay hermetic. * feat(tui): auto-recover the session when the gateway dies unexpectedly When a still-owned gateway child dies while the TUI is alive (a crash, OOM process.exit, or a SIGTERM/SIGHUP forwarded to it), the app currently nulls the session and drops to an inert "gateway exited" state — the user loses a long session and has to restart + re-run everything. That single behavior is most of the "TUI doesn't survive heavy work" complaint, independent of what does the killing. The 'exit' event only reaches this handler on an *unexpected* death: a user /quit calls process.exit before it fires, and a replaced child is identity- skipped in GatewayClient. So on exit we now respawn the gateway and resume the session that was live (history is persisted in SQLite) via a one-shot recoverSidRef the next gateway.ready consults before forging a new session. The in-flight reply is lost (it died with the process) but the session survives. Bounded to GATEWAY_RECOVERY_LIMIT (3) attempts per GATEWAY_RECOVERY_WINDOW_MS (60s) so a gateway that crash-loops on startup can't spawn-storm; past the budget we fall back to the inert state. * fix(tui): sanitize newlines + soften SIGTERM-cause claim in parentLog Address PR review: - recordParentLifecycle collapses embedded \r\n so a multi-line value (e.g. an error message) stays a single breadcrumb and can't masquerade as a separate entry or as the child's panic output sharing the crash log. - Reword the header: a backend SIGTERM is *usually* a parent action but can come straight from an external supervisor (s6, cgroup OOM, stray kill); the presence/absence of a [tui-parent] line before the child's panic is precisely what disambiguates the two. * fix(tui): clear sid during recovery + extract/test the recovery budget Address PR review: - Null `sid` immediately in the gateway exit handler. While the gateway is down (busy=false) the old sid would otherwise let sid-guarded effects (the 1.5s session.active_list poll, queue drain) fire RPCs at a dead/respawning gateway. recoverSidRef carries the session forward; resumeById restores sid on ready. - Extract the respawn budget into a pure evalRecovery() (gatewayRecovery.ts) and unit-test the bound: allows GATEWAY_RECOVERY_LIMIT within the window, blocks past it, and prunes attempts older than the window so recovery re-arms. * fix(tui): cap parent-log breadcrumb length (PR review) Truncate a single persisted breadcrumb to 4096 chars (matching GatewayClient's in-memory log-line cap) so a pathological value — e.g. a giant error string — can't bloat the shared crash log or add noticeable blocking on the synchronous append during a failure path. Covered by a test. * fix(tui): keep "recovering session…" status visible during resume (PR review) resumeById() synchronously sets status to 'resuming…' on entry, so the recovery branch now applies its 'recovering session…' label *after* calling resumeById — the distinct label sticks for the duration of the resume RPC (which later flips to 'ready') instead of being immediately clobbered. Test updated to assert the ordering. * fix(tui): keep recovery budget alive across a startup crash-loop (PR review) deadSid was read from getUiState().sid, which the first exit nulls — so if the respawned gateway crash-looped before gateway.ready (resumeById never restored sid), later exits saw null and abandoned the session after a single attempt, defeating the bounded retry budget. Lift the whole decision into a pure planGatewayRecovery() that falls back to the pending recoverSidRef target when the live sid is already cleared, and unit-test the crash-loop sequence (keeps retrying the same session up to the limit, then falls back to inert). Supersedes evalRecovery. * chore(tui): drop non-null assertion + clarify breadcrumb cap comment (PR review) - Recovery branch guards on `recoverSidRef && recoverSid` so the ref write needs no `!` assertion (avoids a future unsafe refactor). - Reword the parentLog cap comment: it slices the value to 4096 chars and appends a short truncation marker (so the written line is slightly longer), rather than implying a strict 4096-byte limit. * chore(tui): soften "absence ⇒ external signal" + "any in-flight reply" (PR review) - parentLog header: a missing [tui-parent] line only *suggests* an external signal (the logger is best-effort: VITEST-disabled, failed append swallowed), not a definitive conclusion. - Recovery notice says "any in-flight reply was lost" since the gateway can also exit while idle.
…p defaults (NousResearch#35723) * feat(setup): Quick Setup routes through Nous Portal (OAuth + model + messaging) First-time quick setup now goes straight to the Nous Portal provider instead of showing the full provider picker. Runs the device-code OAuth login, selects a Nous model, configures the terminal backend, and offers messaging setup — applying recommended defaults for everything else. - Rename menu entry to 'Quick Setup (Nous Portal)'. - _run_first_time_quick_setup now calls _model_flow_nous (handles both the logged-out OAuth+model-select path and the logged-in curated picker), then re-syncs config from disk to avoid the NousResearch#4172 stale-overwrite. - Terminal / defaults / messaging steps unchanged. * feat(setup): thin out Full Setup with happy defaults Full Setup no longer asks for every config knob — anything with an obvious default is applied silently and stays tunable via the per-section commands (hermes setup agent|terminal|tts, hermes auth add). - Model section: drop the same-provider rotation pool, vision-backend picker, and TTS provider sub-flows. Vision auto-detects from the main provider; TTS defaults to Edge; rotation lives in hermes auth add. - Terminal section: keep the backend picker (Local default) and any required credentials (Modal token, SSH host/user/key, Daytona key), but stop prompting for container image, CPU/mem/disk resources, gateway cwd, and sudo password — all use defaults. - Agent Settings: removed from the wizard. First installs get recommended defaults silently; existing installs keep their tuned values. - New defaults: max_turns 90 -> 150, session_reset both -> none. - Tests: reconfigure tests assert agent settings are no longer prompted on existing installs; drop 3 tests covering the deleted in-setup rotation flow.
…r-user thread (NousResearch#35959) In a per-user thread (thread_sessions_per_user=True), each participant gets an isolated session key (...:{thread_id}:{user_id}). A run another user started lives under a different key, so the caller's own /stop found nothing and replied 'no active task to stop'. When /stop finds no run under the caller's own key, fall back to interrupting any running agent(s) sharing the caller's thread prefix ({chat_id}:{thread_id}), gated on _is_user_authorized. Thread-only — the fallback returns [] for non-thread channels, and a prefix-collision guard prevents thr1 from matching thr11.
…update_cwd authoritative terminal_tool re-sent the init-time/config cwd on every command, clobbering session-local `cd` state: the environment tracked the new directory in `env.cwd`, but foreground/background calls forced the old cwd back. A small `_resolve_command_cwd` resolver now applies the precedence `workdir > live env.cwd > config/override cwd` to: - foreground `env.execute(...)` - background `process_registry.spawn_local(...)` - background `process_registry.spawn_via_env(...)` Additionally, syncing the cwd onto the live cached env when a `cwd` override is (re-)registered. Preferring live `env.cwd` would otherwise demote the ACP `update_cwd` override (registered via `register_task_env_overrides` on `session/load` / `session/resume`) below an already-set `env.cwd`, silently ignoring an editor's mid-session project-root change once any command had run. `register_task_env_overrides` now pushes a new cwd onto the cached env so an explicit ACP cwd change wins, while ordinary in-session `cd` tracking is preserved. Regression coverage: - foreground/background commands follow live `env.cwd` - explicit `workdir` still overrides everything - registering a cwd override updates the live env cwd (ACP authority) - no-op when no live env exists; non-cwd overrides leave env.cwd untouched Based on NousResearch#35510 by @Dusk1e. Co-authored-by: Dusk1e <yusufalweshdemir@gmail.com>
Background tasks on non-local backends (SSH/Docker/Modal/Daytona/Singularity)
go through `ProcessRegistry.spawn_via_env`, which builds a hand-crafted,
shell-safe wrapper:
mkdir -p T && ( nohup bash -lc CMD > LOG 2>&1; rc=$?; ... ) & echo $! > PID && cat PID
`BaseEnvironment.execute()` unconditionally ran `_rewrite_compound_background`
on every command, including this wrapper. The rewrite (meant to defuse the
`A && B &` subshell-wait trap for user commands) turns `( ... ) & echo $!` into
`{ ( ... ) & } echo $!` — note `} echo` with no separator, which is a bash
syntax error. The wrapper then never produces a PID, the redirected output file
is never created, and the agent sees an immediate exit code -1. This breaks
*every* background launch on a non-local backend (e.g. a simple
count-and-redirect script over SSH), not just edge cases.
Fix:
- Add `rewrite_compound_background: bool = True` to `BaseEnvironment.execute()`
(and the `BaseModalExecutionEnvironment` override, which accepts and ignores
it). Default preserves existing behavior; the user foreground terminal path
still rewrites.
- `spawn_via_env` passes `rewrite_compound_background=False` so its already
shell-safe wrapper is left intact.
- Treat a wrapper that produces no PID as a failed launch (mark the session
exited with a real exit code instead of exposing a fake running session), and
don't register/checkpoint a session that never started.
Verified empirically: with the rewrite skipped, the wrapper is valid bash,
launches the process, captures the PID, and writes the log/pid/exit files; the
old rewritten form fails `bash -n` with a syntax error.
Based on NousResearch#33756 by @charzhou (extracted from a multi-feature branch; the
unrelated image_gen / docker-media changes are not included here).
Co-authored-by: CharZhou <17255546+CharZhou@users.noreply.github.com>
…-cwd-acp-aware fix(tools): preserve live session cwd in terminal_tool, keep ACP update_cwd authoritative
…a-env-bg-wrapper fix(tools): don't compound-rewrite spawn_via_env background wrappers
Follow-up to the synthetic-notification DM-topic routing fix. The new _is_telegram_dm_topic_target probed the adapter's _get_dm_topic_info via instance-level getattr, which a MagicMock auto-creates as a truthy callable — so any test double with a non-dm chat_type and a thread_id would be misclassified as a DM topic lane and have the fallback routing keys injected. Resolve the method on type(adapter) and treat only dict-shaped returns as an operator-declared topic, mirroring the existing guard in _rename_telegram_topic_for_session_title. Update the home-channel startup test to declare _get_dm_topic_info on a real adapter subclass instead of patching a MagicMock onto the instance.
Owner
Author
|
Closing this mirror because it used the upstream-target branch and is dirty against fork main. Rebuilt a fork-only mirror from origin/main in fix/dflash-degenerate-final-retry-fork. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
10 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
finish_reason=stopbut ends with degenerate repeated punctuationRoot Cause
The dflash turn did not fail as an empty response or stream timeout. It returned visible text with
finish_reason=stop, so Hermes treated the corrupted final as successful and triggered the background skill-review loop. The skill reviewer then learned from the foreground assistant's unsupported diagnosis.Validation
python3 -m pytest -q -o addopts='' tests/run_agent/test_malformed_final_recovery.py tests/run_agent/test_empty_response_recovery_persistence.py tests/run_agent/test_background_review.py::test_background_review_installs_auto_deny_approval_callback tests/run_agent/test_background_review.py::test_background_review_summary_is_attributed_to_self_improvement_loop tests/run_agent/test_partial_stream_finish_reason.py tests/run_agent/test_provider_fallback.py tests/run_agent/test_retry_status_buffer.pypython3 -m compileall -q run_agent.py agent/conversation_loop.py agent/background_review.py tests/run_agent/test_malformed_final_recovery.pygit diff --checkUpstream PR: NousResearch#36065
Note: local pytest needed
-o addopts=''because this shell lacks the project timeout plugin.