Repository navigation
Conversation
`hermes update` blocks forever on a prompt nobody can answer. The report is a
Windows Scheduled Task with `-WindowStyle Hidden`: 44 minutes parked on
"Restore local changes now? [Y/n]", ending in exit code 0x40010004, an external
kill rather than an exit.
Every existing guard misses it, and not by accident. A hidden console is a real
console, so `sys.stdin.isatty()` is True and both `_non_interactive_update` and
the `prompt_for_restore` predicate classify the run as interactive. The handle
is open, so `input()` never raises EOFError either; the existing
`except (EOFError, UnicodeDecodeError)` guard is not wrong, it is unreachable.
No property of the process separates "console a human is watching" from
"console nobody will ever type into".
So bound the prompts instead of predicting them. An unanswered prompt takes its
documented default, says so, and latches the run as unattended, which is an
observation rather than the inference every isatty check makes.
- `_read_line_with_timeout` reads on a daemon thread and gives up after the
timeout, returning (response, timed_out). `timeout <= 0` restores today's
blocking read, so the old behaviour is reachable from a test rather than only
from a revert. daemon=True is load bearing: the reader stays parked on stdin
for the life of the process, and a non-daemon thread there would block
interpreter shutdown, which is the same unbounded wait wearing a different
hat.
- `_prompt_proven_unattended`, reset at the top of `_cmd_update_impl` so it
scopes to one update rather than to the interpreter. It also closes a hazard
the timeout opens: without it a later prompt could arm a second reader on the
same stdin and have its answer swallowed by the first, still-parked one.
- `_report_unanswered_prompt` names the prompt, the consequence and `--yes`, so
an unattended transcript explains itself instead of ending at a bare prompt.
- `_restore_stashed_changes`: the reported site. Default "n", identical to the
EOFError path it replaces, and safe because the stash stays on disk with its
existing `git stash apply` guidance.
- `_sync_with_upstream_if_needed`: bounded too, though it was not reported. It
runs before the stash restore, so on a fork with no upstream remote an
unattended run hangs there and never reaches the reported hang at all.
- The config-migration prompt consults the latch through the non-interactive
branch it already has. Its "auto" default and everything downstream are
unchanged; only the signal feeding the branch is new.
Deliberately not using `GetConsoleWindow()` + `IsWindowVisible()`. ConPTY hosts
are reported to leave a hidden console window on the process, which would
misclassify Windows Terminal and VS Code users as unattended, and the failure
would be silent. I could not verify that behaviour, and a silent-failure
detector is not something to ship on trust.
Two choices are named for a maintainer rather than made quietly: the 300s value,
and whether a timeout should default to skip-restore (chosen, cannot lose work)
or to restore (matching the `[Y/n]` default a human gets by pressing enter).
16 regression tests. Sabotage proof, five mutations, each caught: removing the
bound fails 6; not latching fails 2; daemon=False fails 1; returning "" instead
of the default fails 2; inverting the blank-line spacing fails 1. The fourth is
why the helper returns an explicit default: both call sites test
`response in {"", "y", "yes"}`, so "" does not mean "no answer", it means yes.
Existing update suites: 15 passed, 7 failed, and the same 7 fail identically on
the unpatched tree (Windows-only console-encoding crashes in `print()`).
Fixes NousResearch#92303
Reviewed What's good
Suggestions
Strong engineering; #1 is process, not product, but it matters here. |
Review follow-up, two points. _report_unanswered_prompt read the module constant while _read_line_with_timeout already accepted a per-prompt timeout, so the first call site to override the bound would have printed a number the run never waited. It now takes the same optional timeout and resolves it the same way, with two tests pinning the default and the override apart; the override test fails against the old hardcoded form. Also documented the abandoned reader's lifetime at the point where it is abandoned. A thread parked on input() cannot be cancelled portably, so it keeps its claim on stdin for the life of the process and a late-typed line races between it and any git subprocess inheriting the same handle. That is accepted rather than fixed, and the comment says why: the run has just produced evidence that nobody is typing, and sending every later subprocess to DEVNULL on the strength of one timeout would change how git behaves in a run that merely paused.
|
All three acted on. 1. #92448. Noted on that PR rather than only here, since coordination that lives on one side of an overlap is not coordination. #92448 (comment) Two things came out of reading their diff properly. The double-report worry does not survive the merge, as far as I can tell. Their guard sits ahead of the read in both functions: The more useful finding is that the two approaches do not overlap as much as the file diff suggests. So: complementary, and I offered to do the rebase from either side depending on which lands first. If theirs goes in ahead of mine, #92410 reduces to the bounding with their guard kept as the fast path. 2. Abandoned reader. Documented at the point where the reader is abandoned. The comment states the race you describe (the parked thread keeps its claim on stdin for the life of the process, so a late-typed line races between it and any git subprocess inheriting the same handle) and says it is accepted rather than fixed, with the reason: the run has just produced evidence that nobody is typing, and routing every later subprocess to 3. Not a nit. 18 passed; |
The hidden-window case (-WindowStyle Hidden) gives a real console so isatty() returns True and this guard does not fire. Document the limitation honestly and point at the bounded-wait approach (NousResearch#92410) as the complement.
_sync_with_upstream_if_needed called bare input() with no assume_yes parameter and no tty check, so a fork checkout without an upstream remote wedged hermes update forever in any non-interactive context (CI, cron, the desktop updater hand-off): stdin stays open, EOFError never fires. Thread assume_yes and the gateway input_fn into the helper and skip the prompt as a decline under assume_yes or a non-tty stdio pair, without writing the decline marker or touching git remotes, so interactive runs still get asked later. Both call sites forward the interaction state; the config-migration and stash-restore prompts already carry this gate. Closes #60240 (prompt half). Supersedes #78678, #92448, #92410. Co-authored-by: BlackishGreen33 <BlackishGreen33@users.noreply.github.com> Co-authored-by: salch-cred <salch-cred@users.noreply.github.com> Co-authored-by: jackulau <jackulau@users.noreply.github.com>
|
The fork-upstream prompt-hang portion of this is superseded by #97052, now merged (00bbfc6): |
_sync_with_upstream_if_needed called bare input() with no assume_yes parameter and no tty check, so a fork checkout without an upstream remote wedged hermes update forever in any non-interactive context (CI, cron, the desktop updater hand-off): stdin stays open, EOFError never fires. Thread assume_yes and the gateway input_fn into the helper and skip the prompt as a decline under assume_yes or a non-tty stdio pair, without writing the decline marker or touching git remotes, so interactive runs still get asked later. Both call sites forward the interaction state; the config-migration and stash-restore prompts already carry this gate. Closes NousResearch#60240 (prompt half). Supersedes NousResearch#78678, NousResearch#92448, NousResearch#92410. Co-authored-by: BlackishGreen33 <BlackishGreen33@users.noreply.github.com> Co-authored-by: salch-cred <salch-cred@users.noreply.github.com> Co-authored-by: jackulau <jackulau@users.noreply.github.com>
What does this PR do?
hermes updatecan block forever on a prompt nobody is able to answer. Thereport is a Windows Scheduled Task with
-WindowStyle Hidden: 44 minutes parkedon
Restore local changes now? [Y/n], ending in exit code0x40010004, which isan external kill rather than an exit. The update neither completed nor failed.
Why every existing guard misses it. A hidden console is still a real
console.
sys.stdin.isatty()returnsTrue, so_non_interactive_updateandthe
prompt_for_restorepredicate both classify the run as interactive:And the handle is open, so
input()never raisesEOFErroreither. Theexisting
except (EOFError, UnicodeDecodeError)guard is not wrong, it is simplynever reached. There is no property of the process that distinguishes "console
a human is watching" from "console nobody will ever type into".
So the prompts are bounded rather than predicted. An unanswered prompt takes
its documented default, says so in the log, and latches the run as unattended.
The latch is the interesting part: a prompt that went unanswered is an
observation that nobody is at the keyboard, which is strictly stronger than
anything
isattycan infer, so later prompts in the same run stop asking.The route I did not take
My first instinct was
GetConsoleWindow()+IsWindowVisible(): a consolenobody can see is a console nobody can type into. I dropped it because ConPTY
hosts (Windows Terminal, the VS Code terminal) are reported to leave a hidden
console window on the process, which would misclassify a large fraction of
Windows users as unattended, and the failure would be silent: their local
changes would just stop being offered for restore.
I could not verify that ConPTY behaviour from where I was working (with stdin
redirected,
GetConsoleWindow()returns0andisatty()is alreadyFalse,which is the case the existing guards already handle). I would rather not ship a
silent-failure detector on a behaviour I am taking on trust. Flagging it in case
a maintainer knows it cold and prefers that route.
Two calls I am handing to a maintainer rather than making quietly
prompt is never affected, short enough that "forever" becomes "five minutes".
It is a judgement call, not a derived number. Happy to change it, or to put it
behind config/env if you want it tunable; I did not add a config key because
that is a new public surface for what is currently one constant.
EOFErrorpath that already exists. It is the only default that cannot losework: the stash stays on disk and the existing
git stash apply <ref>guidance still prints. The argument for the other side is real, though, and I
want it on the record:
[Y/n]means a human pressing enter gets restore, soan argument exists that a timeout should match the visible default. I think
"unattended" is a different situation from "pressed enter" and should be
allowed a more conservative answer, but this is your semantics to set.
Related Issue
Fixes #92303
Type of Change
Changes Made
All in
hermes_cli/update_cmd.py._read_line_with_timeout(default, timeout=None, read_fn=None)(new).Reads one line on a daemon thread and gives up after
timeout, returning(response, timed_out).timeout <= 0restores today's unbounded blockingread, so the old behaviour stays reachable from a test rather than only from a
revert.
daemon=Trueis load bearing: the reader stays parked on stdin for thelife of the process, and a non-daemon thread there would block interpreter
shutdown, which is the same unbounded wait wearing a different hat.
_prompt_proven_unattended(new) plus_reset_prompt_interactivity(),called at the top of
_cmd_update_implso the latch scopes to one updaterather than to the interpreter. It also closes a hazard the timeout would
otherwise open: without it, a later prompt could arm a second reader on the
same stdin and have its answer swallowed by the first, still-parked one.
_report_unanswered_prompt(new). Names the prompt, the consequence, and--yes/--keep-stash, so an unattended transcript explains itself insteadof ending at a bare prompt line. This is the reporter's option 3, folded in.
_restore_stashed_changes: the reported site, now bounded. Default"n",identical to the
EOFErrorpath it replaces._sync_with_upstream_if_needed: bounded too, and this one is worth callingout because it was not in the report. It runs at line 6046 while the stash
restore runs at 6053, so on a fork with no upstream remote an unattended
run hangs here and never reaches the reported hang at all. Fixing only what
was reported would have left the same bug one prompt earlier.
elif _prompt_proven_unattended or not (sys.stdin.isatty() and ...). That isdeliberately routed into the branch that already exists rather than given its
own: the
"auto"default and everything downstream are unchanged, so the onlynew thing is a better signal. This is the generalisation asked for at the end
of the report, without changing any prompt's semantics under cover of a stash
fix.
How to Test
pytest tests/hermes_cli/test_update_prompt_timeout.py -q # 16 passed16 new tests in a new file, covering three things: the bound exists and takes the
safe default; the answered paths are unchanged, including the
EOFError,UnicodeDecodeErrorandKeyboardInterruptpaths that predate this fix; and thelatch.
Sabotage proof. Each mutation applied to
hermes_cli/update_cmd.pyalone,with the new suite re-run:
test_an_unanswerable_prompt_gives_up,test_the_reader_thread_cannot_keep_the_process_alive,test_a_timeout_proves_the_run_unattended,test_a_later_prompt_does_not_ask_again,test_the_reported_hang_now_ends,test_it_stops_waiting_tootest_a_timeout_proves_the_run_unattended,test_a_later_prompt_does_not_ask_againdaemon=Falseon the reader threadtest_the_reader_thread_cannot_keep_the_process_alive""on timeout instead of the defaulttest_an_unanswerable_prompt_gives_up,test_it_stops_waiting_tootest_eof_keeps_the_blank_line_it_always_printedTwo rows worth a second look.
Row 4 is the reason the helper returns an explicit
defaultrather than"". Both call sites testresponse in {"", "y", "yes"}, so an empty stringdoes not mean "no answer", it means yes. A timeout that returned
""wouldhave silently added an upstream remote and silently restored a stash on exactly
the unattended runs this PR exists to make safe, and every "did it time out?"
assertion would still have passed.
Row 5 is a spacing inversion, and it is in here because I made it. Routing
that prompt through the shared helper made it natural to print its blank line on
the answered path instead of the exception path, which is backwards and which
no assertion about a return value would ever notice.
Existing suites.
test_update_yes_flag.py,test_update_autostash.pyandtest_update_streaming.py: 15 passed, 7 failed, 1 skipped, and the same 7 failidentically on the unpatched tree. They are Windows-only console-encoding
crashes (
UnicodeEncodeError: 'charmap' codec can't encode character '✓'out of a
print()), unrelated to this change. Verified by stashing onlyhermes_cli/update_cmd.pyand re-running the same command.Checklist
Code
fix(scope):,feat(scope):, etc.)pytest tests/ -qand all tests passLeaving that box honest rather than ticked:
pytest tests/ -qdoes notcollect on Windows at all.
tests/hermes_cli/test_doctor_journal_modes.pyraisesAttributeError: module 'os' has no attribute 'geteuid'at import and aborts therun. What I did run is the new suite plus every existing suite that touches these
functions, with a stashed baseline for the pre-existing failures, as above. Linux
CI on this PR is the authority for the full-tree line.
Documentation & Housekeeping
docs/, docstrings): the newhelpers carry the reasoning, including why
daemon=Truematters and whythe latch exists
cli-config.yaml.exampleif I added/changed config keys: N/A.The timeout is a module constant, deliberately not a config key yet (see
the maintainer calls above)
CONTRIBUTING.mdorAGENTS.mdif I changed architecture or workflows: N/Athe whole point is that it is not platform-specific.
threadingplus abounded
joinbehaves the same everywhere, which is exactly why I rejectedthe Win32 console-handle route. The bug is reported on Windows but the same
shape reaches CI and remote automation on any OS whenever stdin is a live
handle nobody answers
Screenshots / Logs
Before, from the report:
After: