Skip to content

Studio: fix Ctrl+C shutdown ordering (installer shell + uvicorn thread wait) - #6566

Merged
danielhanchen merged 7 commits into
mainfrom
studio-shutdown-ordering
Jun 22, 2026
Merged

danielhanchen merged 7 commits into
mainfrom
studio-shutdown-ordering

Conversation

@danielhanchen

Copy link
Copy Markdown
Member

Summary

On Ctrl+C, Studio's shutdown logs and the shell prompt interleave, and the auto-start prompt can launch even when declined. This consolidates the two halves of the fix. It supersedes #6564 (installer) and #6565 (server thread) and folds in the run.py-side work from #6565 by @Imagineer99 with two refinements.

There are two independent causes, on Linux / macOS / WSL:

  1. The non-interactive installer shell (curl | sh) dies on SIGINT before the studio child finishes, so the outer shell prompt prints in the middle of the shutdown logs.
  2. The studio process returns to the shell while its uvicorn daemon thread can still write shutdown logs, so they land after the prompt (or get cut off).

Both are needed for a clean shutdown.

1. Installer (install.sh)

The auto-start prompt also had a second bug: declining still launched. The read fallbacks defaulted to yes (read ... || _reply="y" and the no-tty branch), so any answer that was not a cleanly delivered y/n line auto-started a blocking foreground server (closed/EOF tty, or a n typed early and consumed by the earlier package-accept prompt).

  • Default the unreadable cases to n; a real Enter still counts as yes via ${_reply:-y}.
  • trap '' INT before launching Studio so the installer shell waits for Studio's own graceful shutdown instead of dying first and racing the prompt over its logs. The child still receives SIGINT from the terminal.

2. Server thread (studio/backend/run.py, unsloth_cli/commands/studio.py)

From #6565 by @Imagineer99: retain the uvicorn thread (_server_thread) and add _wait_for_server_shutdown(), which joins it and flushes stdout/stderr. Terminal entrypoints call it after requesting shutdown (run.py's main shutdown path and the CLI shutdown/KeyboardInterrupt/BaseException paths) so the prompt cannot return while the thread still owns the streams.

Two refinements on top of #6565:

  • Bound the join at 5s (_SERVER_SHUTDOWN_JOIN_TIMEOUT, matching the per-subprocess timeouts in _graceful_shutdown). The original joined with timeout=None at every call site, which could hang the terminal forever if uvicorn shutdown stalled and left the existing is_alive() warning branch unreachable.
  • Restore SIG_DFL for SIGINT/SIGTERM at the start of the signal handler so a second Ctrl+C force-quits, and drop the redundant in-handler wait (the post-loop wait already covers the signal path). This also keeps the wait out of the signal handler.

Validation

  • bash -n, dash -n, shellcheck clean on install.sh.
  • python -m py_compile and ast.parse clean on run.py, studio.py, the test.
  • pytest tests/test_studio_shutdown_thread_wait.py passes (6 source-level AST tests, including new ones for the bounded join and the SIG_DFL restore).
  • PTY reproductions under a real interactive bash:
    • Installer: declining (n/N/EOF) no longer launches; Enter still launches; the prompt prints after all shutdown logs.
    • Server: with the wait, uvicorn's INFO: Shutting down prints before the prompt (without it, the line is lost). A stalled uvicorn thread exits via the 5s bound (no infinite hang); a second Ctrl+C force-quits in ~0.2s.

Notes

install.ps1 gates the prompt on UserInteractive -and (-not [Console]::IsInputRedirected) and runs Studio in process via & $UnslothExe, so neither issue applies on Windows. No change there.

danielhanchen and others added 2 commits June 22, 2026 11:36
…own logs ordered

The `curl | sh` Studio auto-start prompt had two issues on Linux/macOS/WSL
(install.sh). install.ps1 already gates on input redirection, so Windows is
unaffected.

1. Typing n, or any closed/EOF /dev/tty, still launched Studio. The read
   fallbacks defaulted to "y" (read failure, and the no-tty branch), so any
   answer other than a cleanly delivered y/n line auto-started a blocking
   foreground server. Default those to "n"; a real Enter still counts as yes
   via ${_reply:-y}.

2. On Ctrl+C the shell prompt printed in the middle of Studio's shutdown logs.
   The non-interactive installer shell took the default SIGINT action and died
   before the child finished its graceful shutdown, so the prompt raced ahead
   of "All subprocesses cleaned up". trap '' INT in the installer shell so it
   waits for Studio's own graceful shutdown.
…rl+C

Builds on #6565 by @Imagineer99. The studio server runs uvicorn in a daemon
thread, so on Ctrl+C the process could return to the shell while that thread
was still writing its shutdown logs, interleaving them with the prompt.

Retain the uvicorn thread and join it (flushing stdout/stderr) before terminal
entrypoints return, from run.py's main shutdown path and the CLI shutdown paths.

Refinements over #6565:
- Bound the join at 5s (_SERVER_SHUTDOWN_JOIN_TIMEOUT, matching the existing
  _graceful_shutdown subprocess timeouts) so a stalled uvicorn shutdown cannot
  hang the terminal; the timeout warning branch is now reachable.
- Restore SIG_DFL for SIGINT/SIGTERM at the start of the signal handler so a
  second Ctrl+C force-quits, and drop the redundant in-handler wait (the
  post-loop wait already covers the signal path).

Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request improves the terminal shutdown behavior of Unsloth Studio by ensuring that the shell prompt does not return while the background uvicorn server thread is still writing shutdown logs. It introduces a bounded wait for the server thread, restores default signal handlers upon the first interrupt to allow a force-quit on a second interrupt, and adds regression tests for these behaviors. The review feedback highlights two important improvements: resetting the SIGINT trap in a subshell in install.sh to prevent child processes from inheriting an ignored signal disposition, and restoring SIGBREAK to its default handler on Windows to ensure force-quits work reliably there as well.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread install.sh Outdated
Comment on lines 3145 to 3146
trap '' INT
"$VENV_DIR/bin/unsloth" studio -p 8888 </dev/null

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

When a shell script ignores a signal using trap '' INT, any child process executed by that shell inherits the ignored signal disposition (SIG_IGN). This means unsloth studio and any of its spawned subprocesses (such as llama-server, cloudflared, or training workers) will start with SIGINT ignored. While Python can override this, other non-Python subprocesses may not, leading to orphaned processes or broken graceful shutdowns upon pressing Ctrl+C.

To prevent this, reset the SIGINT trap to its default behavior within a subshell before executing the command.

Suggested change
trap '' INT
"$VENV_DIR/bin/unsloth" studio -p 8888 </dev/null
trap '' INT
(trap - INT; exec "$VENV_DIR/bin/unsloth" studio -p 8888 </dev/null)

Comment thread studio/backend/run.py
Comment on lines +1329 to +1330
signal.signal(signal.SIGINT, signal.SIG_DFL)
signal.signal(signal.SIGTERM, signal.SIG_DFL)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

On Windows, some terminals send SIGBREAK instead of or in addition to SIGINT when Ctrl+C or Ctrl+Break is pressed. Since SIGBREAK is registered to this handler on Windows, we should also restore SIGBREAK to its default disposition (SIG_DFL) to ensure a second Ctrl+Break successfully force-quits the process if the graceful shutdown stalls.

Suggested change
signal.signal(signal.SIGINT, signal.SIG_DFL)
signal.signal(signal.SIGTERM, signal.SIG_DFL)
signal.signal(signal.SIGINT, signal.SIG_DFL)
signal.signal(signal.SIGTERM, signal.SIG_DFL)
if hasattr(signal, "SIGBREAK"):
signal.signal(signal.SIGBREAK, signal.SIG_DFL)

@danielhanchen

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d116d7a417

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread install.sh Outdated
Comment on lines 3145 to 3146
trap '' INT
"$VENV_DIR/bin/unsloth" studio -p 8888 </dev/null

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid inheriting ignored SIGINT into Studio

When the user accepts the auto-start prompt, this trap '' INT is inherited by the foreground unsloth studio process on POSIX shells, so Python starts with SIGINT ignored rather than raising KeyboardInterrupt. The CLI path only catches KeyboardInterrupt and does not install its own SIGINT handler, so Ctrl+C no longer requests Studio's graceful shutdown in the installer-launched server; users are left with a foreground server that ignores Ctrl+C until they close the terminal or send another signal.

Useful? React with 👍 / 👎.

Comment thread studio/backend/run.py
Comment on lines +1329 to +1330
signal.signal(signal.SIGINT, signal.SIG_DFL)
signal.signal(signal.SIGTERM, signal.SIG_DFL)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Restore SIGBREAK for Windows force-quit

On Windows, the handler is also registered for SIGBREAK just below, and the existing comment notes that some terminals deliver Ctrl+C/Ctrl+Break that way. This restore block leaves SIGBREAK pointing at _signal_handler, so in those terminals a second Ctrl+C during a stalled graceful shutdown re-enters the cleanup path instead of taking the default action, defeating the new force-quit escape hatch for that environment.

Useful? React with 👍 / 👎.

- install.sh: run studio in a subshell that resets INT to default
  (trap - INT; exec ...) so the foreground child does not inherit the
  installer shell's ignored SIGINT, which would otherwise swallow the
  studio process's own Ctrl+C and graceful shutdown.
- run.py: also restore SIGBREAK to SIG_DFL in the signal handler so a
  second Ctrl+Break force-quits on Windows, matching SIGINT/SIGTERM.
@danielhanchen

Copy link
Copy Markdown
Member Author

Addressed both review findings in 7e25c27:

  • install.sh: the studio launch now runs in a subshell that resets INT to default, (trap - INT; exec "$VENV_DIR/bin/unsloth" studio -p 8888 </dev/null), so the foreground child no longer inherits the installer shell's ignored SIGINT. Verified under a pty: without it, the in-process CLI path (which relies on the default SIGINT to raise KeyboardInterrupt) swallows Ctrl+C; with it, the child handles Ctrl+C while the installer shell still waits for graceful shutdown, so the prompt stays after the logs.
  • run.py: the signal handler now also restores SIGBREAK to SIG_DFL on Windows, matching SIGINT/SIGTERM, so a second Ctrl+Break force-quits if graceful shutdown stalls.

Validated on Linux, macOS, Windows, and WSL via CI.

danielhanchen added a commit to danielhanchen/unsloth-staging-2 that referenced this pull request Jun 22, 2026
@danielhanchen

Copy link
Copy Markdown
Member Author

@codex review

@danielhanchen

Copy link
Copy Markdown
Member Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request improves the shutdown behavior of Unsloth Studio to ensure the shell prompt does not return while the background server thread is still writing shutdown logs. It introduces a bounded wait for the uvicorn thread, flushes standard streams, and restores default signal handlers on the first interrupt to allow a second interrupt to force-quit. Additionally, install.sh is updated to run the studio in a subshell while ignoring INT in the parent shell. Feedback points out a critical issue in install.sh where set -e is active; if the subshell exits with a non-zero status, the script will terminate immediately instead of capturing the exit code in _LAUNCH_EXIT. A suggestion is provided to safely capture the exit status.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread install.sh Outdated
Comment on lines 3147 to 3149
trap '' INT
(trap - INT; exec "$VENV_DIR/bin/unsloth" studio -p 8888 </dev/null)
_LAUNCH_EXIT=$?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Because set -e is active in install.sh, if the subshell exits with a non-zero status, the shell will terminate immediately and never reach the _LAUNCH_EXIT=$? assignment or the subsequent if block. This prevents the migration warning from being printed when _MIGRATED is true.

Using || _LAUNCH_EXIT=$? prevents set -e from triggering on failure and correctly captures the exit status.

Suggested change
trap '' INT
(trap - INT; exec "$VENV_DIR/bin/unsloth" studio -p 8888 </dev/null)
_LAUNCH_EXIT=$?
trap '' INT
_LAUNCH_EXIT=0
(trap - INT; exec "$VENV_DIR/bin/unsloth" studio -p 8888 </dev/null) || _LAUNCH_EXIT=$?

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🚀

Reviewed commit: 7e25c277f9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@danielhanchen

Copy link
Copy Markdown
Member Author

Good catch on the set -e interaction. install.sh runs under set -e, so a non-zero studio exit from the subshell would abort before _LAUNCH_EXIT=$? and skip the migration hint. Fixed in d75ca27 by capturing with || _LAUNCH_EXIT=$? (initialized to 0 first). Verified under set -e that the migration hint now prints on a non-zero exit and the real exit code is preserved, with a clean exit 0 on success. Re-validated across Linux, macOS, Windows, and WSL.

@danielhanchen

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Already looking forward to the next diff.

Reviewed commit: d75ca27246

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.

@danielhanchen

Copy link
Copy Markdown
Member Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request improves the terminal shutdown behavior of Unsloth Studio by ensuring the uvicorn server thread is properly joined with a timeout, and restoring default signal handlers on interrupt to allow force-quitting. It also updates the installer to default to a non-interactive 'no' when the TTY is unreadable. The review feedback suggests avoiding broad, silent exception handling when flushing streams, and refactoring duplicated shutdown calls in the CLI commands into finally blocks for better maintainability.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread studio/backend/run.py
Comment on lines +688 to +689
except Exception:
pass

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Avoid using silent exception handlers. Instead of silently ignoring the exception with pass, catch the specific exceptions and log them (at least at a debug level) to aid in troubleshooting and future debugging.

Suggested change
except Exception:
pass
except (OSError, ValueError) as e:
logger.debug("Failed to flush stream: %s", e)
References
  1. Avoid using broad, silent exception handlers like except Exception: pass. Instead, log the exception, even if at a debug level, to aid in future debugging.

Comment thread unsloth_cli/commands/studio.py Outdated
Comment on lines +865 to +869
getattr(run_mod, "_wait_for_server_shutdown", lambda: None)()
except KeyboardInterrupt:
run_mod._graceful_shutdown(run_mod._server)
typer.echo("\nShutting down...")
getattr(run_mod, "_wait_for_server_shutdown", lambda: None)()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The call to _wait_for_server_shutdown is duplicated for both the normal shutdown path and the KeyboardInterrupt handler. You can simplify this by using a finally block to ensure this cleanup logic is always executed upon exiting the try block, regardless of how it exits. This will make the code cleaner and more maintainable. Additionally, accessing the shutdown function via getattr ensures the cleanup path is tolerant of partially-constructed or uninitialized modules/attributes.

Suggested change
getattr(run_mod, "_wait_for_server_shutdown", lambda: None)()
except KeyboardInterrupt:
run_mod._graceful_shutdown(run_mod._server)
typer.echo("\nShutting down...")
getattr(run_mod, "_wait_for_server_shutdown", lambda: None)()
except KeyboardInterrupt:
run_mod._graceful_shutdown(run_mod._server)
typer.echo("\nShutting down...")
finally:
getattr(run_mod, "_wait_for_server_shutdown", lambda: None)()
References
  1. Ensure that cleanup or teardown paths tolerate partially-constructed objects or uninitialized attributes by accessing them using getattr instead of direct access.

Comment thread unsloth_cli/commands/studio.py Outdated
Comment on lines +1375 to +1379
getattr(run_mod, "_wait_for_server_shutdown", lambda: None)()
except KeyboardInterrupt:
run_mod._graceful_shutdown(run_mod._server)
typer.echo("\nShutting down...")
getattr(run_mod, "_wait_for_server_shutdown", lambda: None)()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Similar to another part of this file, the call to _wait_for_server_shutdown is duplicated for the normal shutdown path and the KeyboardInterrupt handler. Using a finally block here would avoid repetition and ensure the shutdown logic is always called, improving code clarity and maintainability. Additionally, accessing the shutdown function via getattr ensures the cleanup path is tolerant of partially-constructed or uninitialized modules/attributes.

Suggested change
getattr(run_mod, "_wait_for_server_shutdown", lambda: None)()
except KeyboardInterrupt:
run_mod._graceful_shutdown(run_mod._server)
typer.echo("\nShutting down...")
getattr(run_mod, "_wait_for_server_shutdown", lambda: None)()
except KeyboardInterrupt:
run_mod._graceful_shutdown(run_mod._server)
typer.echo("\nShutting down...")
finally:
getattr(run_mod, "_wait_for_server_shutdown", lambda: None)()
References
  1. Ensure that cleanup or teardown paths tolerate partially-constructed objects or uninitialized attributes by accessing them using getattr instead of direct access.

@danielhanchen

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Nice work!

Reviewed commit: 683c6e6c31

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@danielhanchen

Copy link
Copy Markdown
Member Author

Applied the finally refactor in d65f401: both CLI shutdown paths (studio_default and run) now call _wait_for_server_shutdown once in a finally block instead of duplicating it across the normal and KeyboardInterrupt paths. Cleaner, and it also runs the wait on any other exit. Kept the getattr access for partial-module tolerance.

Left _flush_standard_streams's except Exception: pass as-is: it is an intentional best-effort flush on the shutdown path, consistent with the other except Exception: pass helpers in run.py, and a debug log there would mostly add noise while the logger is tearing down.

@danielhanchen

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Another round soon, please!

Reviewed commit: d65f401219

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@danielhanchen
danielhanchen merged commit dbc13f0 into main Jun 22, 2026
47 of 57 checks passed
@danielhanchen
danielhanchen deleted the studio-shutdown-ordering branch June 22, 2026 14:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant