Repository navigation
feat: add Brev Launchable provisioning script #687
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 7 commits
Commits
Show all changes
10 commits
Select commit
Hold shift + click to select a range
ab8bb1f
feat: add Brev Launchable provisioning script
zywind 5622c59
fix(brev): address review findings in launchable setup
zywind 298dfab
fix(brev): guard notebook install on both ipykernel and ipywidgets
zywind 1056721
fix(brev): derive package indexes from the installed release
zywind 454d736
feat(brev): show a setup-in-progress notice and move welcome text to …
zywind b505549
docs: add Brev Launchable badge to README and docs
zywind 0dffb0c
fix(brev): gate tutorial fetch on a completion marker
zywind dc517c2
fix(brev): require at least three CUDA indexes, not exactly three
zywind 2824645
refactor(brev): compress setup.sh comments to reclaim script budget
zywind 6358cac
fix(brev): harden CUDA index discovery
zywind File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,190 @@ | ||
| <!-- SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. --> | ||
| <!-- SPDX-License-Identifier: Apache-2.0 --> | ||
|
|
||
| ### NeMo Safe Synthesizer Brev Launchable | ||
|
coderabbitai[bot] marked this conversation as resolved.
Resolved
|
||
|
|
||
| This directory holds the provisioning script and recorded console settings for the | ||
| [NVIDIA Brev](https://brev.nvidia.com) Launchable, a one-click GPU environment for trying | ||
| NeMo Safe Synthesizer without setting up CUDA, drivers, or Python locally. | ||
|
|
||
| Nothing in this directory is executed by the repo or by CI. A Brev Launchable is | ||
| configured in the Brev web console, and the setup script is pasted into a form field | ||
| there. This directory exists so that configuration is versioned and reviewable rather | ||
| than living only in a browser. When you change `setup.sh`, you must also paste the new | ||
| contents into the console for the change to take effect. | ||
|
|
||
| ### Files | ||
|
|
||
| - `setup.sh`: Pasted into the Launchable's Setup Script field. Installs the CUDA | ||
| build of Safe Synthesizer into a dedicated venv, registers it as the default Jupyter | ||
| kernel, and drops the tutorial notebooks in `$HOME`. | ||
| - `welcome.md`: Becomes the customer's `$HOME/README.md`. Fetched at provisioning | ||
| time from the same tarball as the tutorials, not baked into `setup.sh` -- it would | ||
| otherwise consume a tenth of the 16 KiB script budget. | ||
|
|
||
| ### Console configuration | ||
|
|
||
| Recreate the Launchable at <https://brev.nvidia.com> → Launchables → Create | ||
| Launchable with these settings. | ||
|
|
||
| | Section | Field | Value | | ||
| | ------- | ----- | ----- | | ||
| | Software | Runtime mode | VM Mode | | ||
| | Software | Install Jupyter on the host | Enabled | | ||
| | Software | Run a Setup Script | Enabled, contents of `setup.sh` | | ||
| | Software | Image ID | Leave blank | | ||
| | Source | Code source | No code files (`setup.sh` downloads the tutorials itself) | | ||
| | Hardware | GPU | 1× 80 GiB VRAM, single GPU | | ||
| | Hardware | Disk | 200 GiB or more -- not resizable after creation | | ||
| | Network | Ports | 8888, named `jupyter` | | ||
| | Access | Visibility | Anyone with the link | | ||
|
|
||
| Multi-GPU instances are pointless here: training is single-GPU only, so a second GPU | ||
| adds cost and no capability. | ||
|
|
||
| ### Launch parameters | ||
|
|
||
| All optional. Per Brev's guidance, do not put credential values in parameter defaults. | ||
|
|
||
| | Name | Type | Purpose | | ||
| | ---- | ---- | ------- | | ||
| | `NSS_INFERENCE_KEY` | Text | NVIDIA NIM key for column classification. Without it, classification runs in degraded mode. | | ||
| | `HF_TOKEN` | Text | Only needed for gated Hugging Face models. | | ||
|
|
||
| Brev passes these to `setup.sh` as environment variables. | ||
|
|
||
| Deliberately kept to two. A dataset-URL parameter and a model-prewarm toggle were both | ||
| tried and removed: a customer can fetch a URL with one `!curl` line in a notebook, and | ||
| prewarming moves the model download earlier without making it shorter, while hiding the | ||
| progress bar the notebook would otherwise show. Every parameter is visible to every | ||
| deployer, so the bar for adding one is high. | ||
|
|
||
| ### What the customer gets | ||
|
|
||
| `$HOME` is the JupyterLab file browser root, so only what a customer needs is visible | ||
| there. Everything operational is a dotfile, which the browser hides by default. | ||
|
|
||
| ```text | ||
| $HOME/ | ||
| tutorials/ the three tutorial notebooks and their datasets | ||
| README.md where to start, rendered on double-click | ||
| (SETUP-IN-PROGRESS.md until setup finishes) | ||
|
|
||
| .nss-venv/ cu129 venv, registered as the default kernel | ||
| .cache/huggingface/ model cache (Hugging Face's default location) | ||
| .nss-env.sh PATH and VIRTUAL_ENV, sourced from .bashrc | ||
| .nss-setup.log full provisioning log | ||
| ``` | ||
|
|
||
| The script deliberately does not create a data folder. Customers put their files | ||
| wherever they like and pass the path to `with_data_source()`. Bring-your-own-data | ||
| options, in rough order of convenience: drag and drop into the JupyterLab file browser; | ||
| `!curl -O <url>` in a notebook cell; or `brev shell` plus `scp` for large files. | ||
|
|
||
| ### Implementation notes | ||
|
|
||
| These are the non-obvious constraints the script works around. They were each found the | ||
| hard way on a real instance. | ||
|
|
||
| - Package indexes are derived at runtime, not hardcoded. Index URLs are | ||
| install-time configuration rather than wheel metadata, so the installer must supply | ||
| them -- and they have to match the release being installed, not this repo's `main`. | ||
| The script resolves the latest version from the PyPI JSON API, fetches that tag's | ||
| `pyproject.toml`, and reads the CUDA index URLs out of it, then pins the install to | ||
| that exact version so the two cannot drift. Selection is keyed on the URL containing | ||
| `cu129`, not on the index name: the flashinfer entry was renamed | ||
| `flashinfer-jit-cache` → `flashinfer-jit-cache-cu129` between 0.1.8 and 0.1.9, so | ||
| names are not stable across releases. The parse runs inside a process substitution and | ||
| therefore cannot fail the script, so the count of discovered indexes is what validates | ||
| it. | ||
| - uv is installed from a checksum-verified tarball, not `curl | sh`. The | ||
| `astral.sh/install.sh` path logs `no checksums to verify`, so nothing validated what | ||
| it downloaded. The script fetches the pinned release tarball, compares it against the | ||
| published `.sha256`, and installs the binaries -- matching the GPG-verified mise | ||
| install in `tools/install-mise.sh`. | ||
| - A placeholder file marks the home directory while setup runs. JupyterLab | ||
| accepts connections well before this script finishes, so a user who opens it early | ||
| would otherwise see an empty or half-populated file browser and assume the Launchable | ||
| is broken. `SETUP-IN-PROGRESS.md` is written before any slow work, rewritten by the | ||
| `ERR` trap if provisioning fails, and replaced by `README.md` on success. | ||
| - The welcome text lives in `welcome.md`, not a heredoc. It is pulled from the | ||
| same tarball as the tutorials, so the two always match, and it is staged as a dotfile | ||
| until the final step so it never appears while setup is still running. The fetch is | ||
| non-fatal: `script/brev/` exists in no released tag, so it resolves only from the | ||
| `main` fallback until a release includes it. | ||
| - The setup script has a 16 KiB limit. Brev rejects anything larger, which is why | ||
| the script carries short comments pointing here rather than full explanations. Check | ||
| `wc -c script/brev/setup.sh` before pasting. | ||
| - The script runs unprivileged. Brev executes it as the instance user, not root, so | ||
| `/usr/local/bin`, `/var/log`, and `/etc/profile.d` are all unwritable. Everything | ||
| installs under `$HOME`. | ||
| - The venv is deliberately separate from Brev's. Installing `[cu129,engine]` into | ||
| `$HOME/.venv` would resolve torch, vllm, and flashinfer alongside jupyterlab's own | ||
| pins. If that breaks the notebook server, the customer's only interface is gone and | ||
| there is no error they can act on. A separate venv keeps a failed install recoverable. | ||
| - The instance username varies by provider -- `shadeform` on the Shadeform-brokered | ||
| offerings, likely `ubuntu` on direct ones. The script never names an account; it uses | ||
| `$HOME` and `id -un`. | ||
| - The image ships its own `$HOME/.venv` on Python 3.12, which is what the Brev | ||
| Jupyter server runs in. `uv` discovers it by walking up from the working directory, so | ||
| the script pins `VIRTUAL_ENV` in both the kernelspec and `.nss-env.sh` to keep | ||
| `uv pip install` from silently targeting it. Do not delete or upgrade that venv -- | ||
| breaking it breaks the notebook server, which is the customer's only interface. | ||
| - The kernel is registered as `python3`, not a custom name, because the tutorial | ||
| notebooks declare `kernelspec.name = "python3"`. Any other name would force the | ||
| customer to switch kernels by hand. | ||
| - Kernel precedence is counterintuitive. `jupyter_core` puts the server's own | ||
| environment *ahead* of `~/.local/share/jupyter` whenever the server runs inside a | ||
| virtualenv (`prefer_environment_over_user()`), and Brev's does. Registering only at | ||
| user scope is silently shadowed by Brev's 3.12 kernel. The script asks the server's | ||
| interpreter for `jupyter_path("kernels")[0]` and writes there, backing up any existing | ||
| `kernel.json` to `kernel.json.orig`. | ||
| - The kernelspec holds secrets, so it stays mode 0600 inside `$HOME`. An earlier | ||
| version also mirrored it to `/usr/local/share/jupyter` at 0644 via sudo, which would | ||
| have put `NSS_INFERENCE_KEY` and `HF_TOKEN` in a world-readable file. Do not | ||
| reintroduce that -- it also could not win precedence over a venv-scoped kernel anyway. | ||
| - Most providers do not support stop/start. The instance bills from creation until | ||
| deletion, with no pause. Nebius was the exception at the time of writing. | ||
| - Tutorials arrive via tarball, not `git clone`. `--strip-components` lands them flat | ||
| in `tutorials/` rather than the repo's `docs/tutorials/` nesting, and the image is not | ||
| required to have `git`. The tarball is pinned to the tag matching the installed wheel, | ||
| falling back to `main`, so the notebooks can never call APIs newer than the package | ||
| they run against. | ||
| - The tar member is an exact path, not a glob. GNU tar does not enable wildcards for | ||
| member names by default, and bsdtar rejects `--wildcards` outright, so no single glob | ||
| form works on both. The archive's top-level directory name also varies by ref | ||
| (`Safe-Synthesizer-0.1.8` for a tag, `Safe-Synthesizer-main` for a branch), so the | ||
| script reads it from the archive rather than guessing. | ||
| - Never pipe `tar -t` into `head`. Closing the pipe early makes GNU tar fail with | ||
| `stdout: write error` on SIGPIPE, and `pipefail` turns that into a failed run. bsdtar | ||
| exits silently on SIGPIPE, so this passes on macOS and fails on the instance. The | ||
| script reads the listing into a variable and takes the prefix with `${listing%%/*}`. | ||
|
|
||
| ### Verification | ||
|
|
||
| After deploying, check `$HOME/.nss-setup.log` for the `setup complete` banner, then in a | ||
| notebook confirm the environment resolved correctly: | ||
|
|
||
| ```python | ||
| import sys | ||
| print(sys.executable) # expect $HOME/.nss-venv/bin/python | ||
| ``` | ||
|
|
||
| That must hold without touching the kernel picker -- the customer should never have | ||
| to switch kernels. If it reports a 3.12 path instead, kernel registration landed in a | ||
| directory the server does not consult; check the `registered kernel at ...` lines in the | ||
| log against `jupyter kernelspec list`. | ||
|
|
||
| Then run `tutorials/safe-synthesizer-101.ipynb` end to end -- roughly 15 minutes on | ||
| an A100 or H100. | ||
|
|
||
| ### Tested configurations | ||
|
|
||
| | Date | Provider | GPU | Result | | ||
| | ---- | -------- | --- | ------ | | ||
| | 2026-07-29 | Shadeform-brokered | H100 PCIe 80 GiB | uv install, venv creation, `[cu129,engine]` install, tutorial fetch | | ||
|
|
||
| Add a row when you validate another provider or GPU class. The A100-or-larger figure in | ||
| the main docs looks conservative for the default 3B model; if a 48 GiB card proves | ||
| sufficient, record it here and correct the requirement in `README.md`, | ||
| `docs/index.md`, and `docs/user-guide/getting-started.md`. | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.