Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .agents/skills/diagnose-failures/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ Common `ty` error patterns:

| Error | Likely cause | Fix |
|-------|-------------|-----|
| `unresolved-import` | Missing extra in venv | Run `uv sync --frozen --extra cu128 --extra engine --group dev` |
| `unresolved-import` | Missing extra in venv | Run `uv sync --frozen --extra cu130 --extra engine --group dev` |
| `unresolved-attribute` | Computed property treated as config field | Check if the attribute is a `@property`, not a Pydantic field |
| `possibly-unbound` | Variable assigned only in one branch | Add an `else` branch or initialise before the conditional |
| `invalid-argument-type` | Wrong type passed to function | Check the function signature; use `cast()` only as a last resort |
Expand Down Expand Up @@ -87,8 +87,8 @@ gh run view <run-id> --log-failed

## Import / Dependency Errors

- Check if the import requires an extras gate: `cpu`, `cu128`, or `engine`
- Common: `vllm`, `torch`, `unsloth` need `cpu` or `cu128` extra
- Check if the import requires an extras gate: `cpu`, `cu130`, or `engine`
- Common: `vllm`, `torch`, `unsloth` need `cpu` or `cu130` extra
- Use the `diagnose-deps` skill for lockfile diff diagnosis after `uv lock`
- Run: `uv run tools/diff-lockfile.py` to see what changed

Expand All @@ -97,7 +97,7 @@ gh run view <run-id> --log-failed
| Error | Likely Cause | Fix |
|-------|-------------|-----|
| `CUDA out of memory` | Batch too large or model too big | Reduce `batch_size` or use quantization |
| `CUDA not available` | Wrong extra installed | Reinstall with `make bootstrap-nss cu128` |
| `CUDA not available` | Wrong extra installed | Reinstall with `make bootstrap-nss cu130` |
| `NCCL error` | Multi-GPU issues | Use `CUDA_VISIBLE_DEVICES=0` for single-GPU |

### Running GPU / e2e Tests
Expand Down
4 changes: 2 additions & 2 deletions .agents/skills/git-worktrees/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,10 @@ uv sync --frozen

This creates a local `.venv` in the worktree. With uv's cache the install takes ~2-3 seconds on a warm cache.

If you need different extras (e.g. `cu128` vs `cpu`), pass them explicitly:
If you need different extras (e.g. `cu130` vs `cpu`), pass them explicitly:

```bash
uv sync --frozen --extra cu128 --extra engine --group dev
uv sync --frozen --extra cu130 --extra engine --group dev
```

Never run bare `uv sync` without `--frozen` -- it re-locks `uv.lock` and creates dirty state.
Expand Down
10 changes: 5 additions & 5 deletions .agents/skills/uv-build/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ make bootstrap-tools && make bootstrap-nss cpu
# Pick a variant:
make bootstrap-nss dev # dev tools only (no engine/torch)
make bootstrap-nss cpu # + engine + CPU PyTorch
make bootstrap-nss cu128 # + engine + CUDA 12.8 PyTorch
make bootstrap-nss cu130 # + engine + CUDA 13.0 PyTorch
make bootstrap-nss engine # + engine (no torch)
```

Expand All @@ -30,11 +30,11 @@ Under the hood: `uv sync --frozen --extra <extra> [--extra engine] --group dev`
| Extra | What it installs |
|-------|------------------|
| `cpu` | PyTorch CPU, faiss-cpu, flashinfer (Linux only) |
| `cu128` | PyTorch+CUDA 12.8, faiss-gpu, flashinfer-jit-cache |
| `cu130` | PyTorch+CUDA 13.0, faiss-gpu, flashinfer-jit-cache |
| `engine` | ML pipeline deps (outlines, wandb, tiktoken, etc.) -- no torch |
| `microservices` | `nemo-microservices` from local path |

`cpu` and `cu128` conflict -- you must pick one, never both. Enforced in `[tool.uv] conflicts`.
`cpu` and `cu130` conflict -- you must pick one, never both. Enforced in `[tool.uv] conflicts`.

## Index Management

Expand All @@ -43,9 +43,9 @@ PyTorch wheels come from dedicated indexes, not PyPI:
| Index | URL | Used for |
|-------|-----|----------|
| `pytorch-cpu` | `download.pytorch.org/whl/cpu` | torch, torchvision (CPU, Linux) |
| `pytorch-cu128` | `download.pytorch.org/whl/cu128` | torch, torchvision, triton, xformers (CUDA) |
| `pytorch-cu130` | `download.pytorch.org/whl/cu130` | torch, torchvision, triton (CUDA) |
| `nv-shared-pypi-local` | NVIDIA Artifactory | Internal NVIDIA packages |
| `flashinfer-jit-cache` | `flashinfer.ai/whl/cu128` | FlashInfer JIT cache |
| `flashinfer-jit-cache` | `flashinfer.ai/whl/cu130` | FlashInfer JIT cache |
| `nvidia-pypi-public` | `pypi.nvidia.com` | Public NVIDIA packages |

All indexes are `explicit = true` (only used when a package is mapped to them in `[tool.uv.sources]`).
Expand Down
4 changes: 2 additions & 2 deletions .claude/commands/bootstrap.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,9 @@ Set up the development environment from scratch.
2. Install Python dependencies (choose one):
```bash
make bootstrap-nss cpu # CPU-only (macOS or Linux without GPU)
make bootstrap-nss cuda # CUDA 12.8 (Linux with NVIDIA GPU)
make bootstrap-nss cuda # CUDA 13.0 (Linux with NVIDIA GPU)
make bootstrap-nss engine # Engine dependencies only (no torch)
make bootstrap-nss dev # Minimal dev dependencies only
```

Note: `cuda` is an alias for `cu128`. Both are equivalent.
Note: `cuda` is an alias for `cu130`. Both are equivalent.
4 changes: 2 additions & 2 deletions .cursor/setup-worktree.sh
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,10 @@ fi

# Bare --frozen installs the base environment. For GPU dev work (ty, import
# checks, GPU tests) run the full command manually after setup:
# uv sync --frozen --extra cu128 --extra engine --group dev
# uv sync --frozen --extra cu130 --extra engine --group dev
uv sync --frozen
echo "Venv ready: $(pwd)/.venv"
echo "Note: for GPU extras run: uv sync --frozen --extra cu128 --extra engine --group dev"
echo "Note: for GPU extras run: uv sync --frozen --extra cu130 --extra engine --group dev"

for _envfile in .env .env.local mise.local.toml .local.envrc; do
if [ -f "$ROOT_WORKTREE_PATH/$_envfile" ]; then
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/gpu-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ jobs:
bootstrap-tools: "true"

- name: Bootstrap CUDA environment
run: make bootstrap-nss cu128
run: make bootstrap-nss cu130

- name: Check GPU availability
run: |
Expand Down Expand Up @@ -114,7 +114,7 @@ jobs:
bootstrap-tools: "true"

- name: Bootstrap CUDA environment
run: make bootstrap-nss cu128
run: make bootstrap-nss cu130

- name: Check GPU availability
run: |
Expand Down
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ Common commands: `make test` (unit tests), `make format` (auto-fix formatting +
The canonical `uv sync` command for a full GPU/dev environment is:

```bash
uv sync --frozen --extra cu128 --extra engine --group dev
uv sync --frozen --extra cu130 --extra engine --group dev
```

Bare `uv sync --frozen` (without extras) installs an incomplete environment -- `ty`, import checks, and GPU tests will fail.
Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ Please read our [Code of Conduct](CODE_OF_CONDUCT.md) before contributing.

# Install Python dependencies (choose one)
make bootstrap-nss cpu # CPU-only (macOS or Linux without GPU)
make bootstrap-nss cuda # CUDA 12.8 (Linux with NVIDIA GPU)
make bootstrap-nss cuda # CUDA 13.0 (Linux with NVIDIA GPU)
make bootstrap-nss engine # Engine dependencies only
make bootstrap-nss dev # Minimal dev dependencies only
```
Expand Down
14 changes: 7 additions & 7 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ NSS_ROOT_PATH := $(shell pwd)
# Normalize architecture names
ifeq ($(ARCH),x86_64)
ARCH := amd64
PYTORCH_DEPS := cu128
PYTORCH_DEPS := cu130
export BUILD_ARCH ?= linux/amd64
endif
ifeq ($(ARCH),aarch64)
Expand Down Expand Up @@ -82,12 +82,12 @@ verify-python-version: ## Verify Python version and install if necessary
uv venv --seed --allow-existing --python 3.11

.PHONY: bootstrap-python
bootstrap-python: .venv ## Bootstrap Python dependencies. Set PYTORCH_DEPS to 'cpu' or 'cu128'. Here mostly for legacy usage.
bootstrap-python: .venv ## Bootstrap Python dependencies. Set PYTORCH_DEPS to 'cpu' or 'cu130'. Here mostly for legacy usage.
uv sync --frozen --extra ${PYTORCH_DEPS} --extra engine --group dev

# Dynamic targets for bootstrap-nss
# Usage: make bootstrap-nss {dev,engine,cpu,cuda}
BOOTSTRAP_EXTRAS := dev engine cpu cuda cu128
BOOTSTRAP_EXTRAS := dev engine cpu cuda cu130
$(BOOTSTRAP_EXTRAS):
@:

Expand All @@ -97,9 +97,9 @@ bootstrap-nss: .venv ## Bootstrap Python dependencies. Usage: make bootstrap-nss
@echo "~~~~~~"
@echo "attempting to install nss package with primary extra: $(EXTRA)"
@if [ "$(EXTRA)" = "cuda" ]; then \
uv sync --frozen --extra cu128 --extra engine --group dev; \
elif [ "$(EXTRA)" = "cu128" ]; then \
uv sync --frozen --extra cu128 --extra engine --group dev; \
uv sync --frozen --extra cu130 --extra engine --group dev; \
elif [ "$(EXTRA)" = "cu130" ]; then \
uv sync --frozen --extra cu130 --extra engine --group dev; \
elif [ "$(EXTRA)" = "cpu" ]; then \
uv sync --frozen --extra cpu --extra engine --group dev; \
elif [ "$(EXTRA)" = "engine" ]; then \
Expand Down Expand Up @@ -469,7 +469,7 @@ NSS_DATASETS := clinc_oos dow_jones_index

define nss_combo_test
test-nss-$(1)-$(2)-ci: ## Run pytest test for $(shell echo $(1) | tr '_' '-') config with $(shell echo $(2) | tr '_' '-') dataset
$(MAKE) bootstrap-nss cu128
$(MAKE) bootstrap-nss cu130
$(PYTEST_NO_XDIST_CMD) -vv $(PYTEST_CI_OPTS) $(NSS_ROOT_PATH)/tests/e2e/test_dataset_config.py -k "test_$(2)_dataset[$(subst _,-,$(1))]"
endef

Expand Down
12 changes: 6 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,15 +18,15 @@ Read detailed usage below, or jump to the documentation with [Getting Started](h

```bash
# With uv (recommended):
uv pip install "nemo-safe-synthesizer[cu128,engine]" \
--index https://flashinfer.ai/whl/cu128 \
--index https://download.pytorch.org/whl/cu128 \
uv pip install "nemo-safe-synthesizer[cu130,engine]" \
--index https://flashinfer.ai/whl/cu130 \
--index https://download.pytorch.org/whl/cu130 \
--index-strategy unsafe-best-match

# With pip:
pip install "nemo-safe-synthesizer[cu128,engine]" \
--extra-index-url https://download.pytorch.org/whl/cu128 \
--extra-index-url https://flashinfer.ai/whl/cu128
pip install "nemo-safe-synthesizer[cu130,engine]" \
--extra-index-url https://download.pytorch.org/whl/cu130 \
--extra-index-url https://flashinfer.ai/whl/cu130
```

Or install from source:
Expand Down
10 changes: 5 additions & 5 deletions containers/Dockerfile.cuda
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
#
# Stages:
# tools -- install mise and all dev tools (uv, ruff, etc.)
# deps -- install Python, uv, and all cu128+engine dependencies
# deps -- install Python, uv, and all cu130+engine dependencies
# runtime -- minimal CLI wrapper (ENTRYPOINT entrypoint.sh -> safe-synthesizer)
# dev -- extends runtime with dev tools, tests, and interactive shell
#
Expand Down Expand Up @@ -58,7 +58,7 @@ ARG TARGETARCH
# ---------------------------------------------------------------------------
# Build arguments
# ---------------------------------------------------------------------------
ARG CUDA_VERSION=12.8.1
ARG CUDA_VERSION=13.0.2
ARG UBUNTU_VERSION=22.04
# Use "runtime" when all deps ship pre-built wheels (current state).
# Switch to "devel" if a future dependency requires CUDA compilation.
Expand Down Expand Up @@ -142,14 +142,14 @@ RUN --mount=type=cache,target=/root/.cache/uv \
# not when source code changes.
COPY pyproject.toml uv.lock ./
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --no-install-project --extra cu128 --extra engine --no-group dev
uv sync --no-install-project --extra cu130 --extra engine --no-group dev

# -- Project layer: install the project (non-editable so the venv is
# self-contained and source code is not needed in the runtime image).
COPY README.md ./
COPY src/ src/
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --no-editable --extra cu128 --extra engine --no-group dev
uv sync --no-editable --extra cu130 --extra engine --no-group dev

# ---------------------------------------------------------------------------
# Stage 3: runtime -- minimal image wrapping the safe-synthesizer CLI
Expand Down Expand Up @@ -233,7 +233,7 @@ COPY . .

# Install dev dependencies into the existing venv.
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --extra cu128 --extra engine --group dev
uv sync --extra cu130 --extra engine --group dev

RUN git config --global --add safe.directory /workspace

Expand Down
8 changes: 4 additions & 4 deletions docs/developer-guide/docker.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ flowchart TD

subgraph stages [Build Stages]
tools["tools\nInstalls mise + all dev tools\n(.mise.toml is single source of truth)"]
deps["deps\nInstalls Python 3.11 via uv\nuv sync cu128+engine"]
deps["deps\nInstalls Python 3.11 via uv\nuv sync cu130+engine"]
runtime["runtime\nCopies venv + Python\nNon-root appuser\ntini + entrypoint.sh"]
dev["dev\nExtends runtime\nCopies mise tree from tools\nRoot user"]
end
Expand All @@ -44,7 +44,7 @@ flowchart TD
Uses the [mise Docker cookbook](https://mise.jdx.dev/mise-cookbook/docker.html)
pattern with `MISE_DATA_DIR=/mise` for stable, copyable paths.
- deps: copies the uv binary from `tools`, then installs Python and all
cu128+engine dependencies. Uses `--mount=type=cache` to avoid
cu130+engine dependencies. Uses `--mount=type=cache` to avoid
re-downloading ~10 GB of PyTorch/CUDA wheels.
- runtime: copies the venv and uv-managed Python into a fresh CUDA runtime
base. Runs as non-root `appuser` (uid 1000). GPU access is declared via
Expand Down Expand Up @@ -262,8 +262,8 @@ To reduce size:

| Aspect | `Dockerfile.cuda` | `Dockerfile.test_ci` |
|--------|-------------------|----------------------|
| Base | `nvidia/cuda:12.8.1-runtime-ubuntu22.04` | `python:3.11-slim` |
| Extras | `cu128` + `engine` | `cpu` + `engine` |
| Base | `nvidia/cuda:13.0.2-runtime-ubuntu22.04` | `python:3.11-slim` |
| Extras | `cu130` + `engine` | `cpu` + `engine` |
| GPU | Required | Not needed |
| Stages | `tools` / `deps` / `runtime` / `dev` | `setup` / `install-deps` |
| Use case | Training, generation, evaluation | CPU-only unit tests and CI checks |
Expand Down
2 changes: 1 addition & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ NeMo Safe Synthesizer creates private, safe versions of sensitive tabular datase
- Flexible interfaces -- CLI for scripting, Python SDK for programmatic workflows, YAML configuration

!!! info "System Requirements"
NeMo Safe Synthesizer requires a Linux machine with an NVIDIA GPU (A100 80GB+ recommended) and CUDA 12.8+ to run the training and generation pipeline. macOS, Windows, and Apple Silicon are not supported for pipeline execution. A CPU-only install is available for development and configuration validation -- see [Getting Started](user-guide/getting-started.md#install-the-package).
NeMo Safe Synthesizer requires a Linux machine with an NVIDIA GPU (A100 80GB+ recommended) and CUDA 13.0+ to run the training and generation pipeline. macOS, Windows, and Apple Silicon are not supported for pipeline execution. A CPU-only install is available for development and configuration validation -- see [Getting Started](user-guide/getting-started.md#install-the-package).

## Next Steps

Expand Down
2 changes: 1 addition & 1 deletion docs/user-guide/docker.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ for training, generation, and evaluation.

- Docker 20.10+ (BuildKit enabled by default in 23.0+)
- [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) installed and configured
- NVIDIA driver compatible with CUDA 12.8
- NVIDIA driver compatible with CUDA 13.0
- NVIDIA GPU (A100 or better recommended)

Verify GPU access works:
Expand Down
18 changes: 9 additions & 9 deletions docs/user-guide/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,11 @@ does at each stage.
### Prerequisites

- Python 3.11–3.13 (dev tooling currently pins 3.11 via `.python-version` in the repo root; Python 3.14+ is **not** supported — see [Troubleshooting](troubleshooting.md#python-314-is-not-supported))
- CUDA runtime 12.8+
- CUDA runtime 12.9+
- NVIDIA GPU (A100 or larger) for training and generation

!!! failure "Linux only -- macOS, Windows, and Apple Silicon are not supported"
NeMo Safe Synthesizer requires a Linux machine with an NVIDIA GPU and CUDA 12.8+
NeMo Safe Synthesizer requires a Linux machine with an NVIDIA GPU and CUDA 13.0+
to run the training and generation pipeline. The [CPU install tab below](#install-the-package)
is for development and configuration validation only -- it cannot train models or
generate synthetic data.
Expand All @@ -29,22 +29,22 @@ does at each stage.
The CUDA and CPU extras depend on packages (PyTorch, FlashInfer) hosted on
indexes outside PyPI. You must pass the extra index URLs shown below.

=== "CUDA 12.8 (Linux with NVIDIA GPU)"
=== "CUDA 13.0 (Linux with NVIDIA GPU)"

=== "pip"

```bash
pip install "nemo-safe-synthesizer[cu128,engine]" \
--extra-index-url https://download.pytorch.org/whl/cu128 \
--extra-index-url https://flashinfer.ai/whl/cu128
pip install "nemo-safe-synthesizer[cu130,engine]" \
--extra-index-url https://download.pytorch.org/whl/cu130 \
--extra-index-url https://flashinfer.ai/whl/cu130
```

=== "uv"

```bash
uv pip install "nemo-safe-synthesizer[cu128,engine]" \
--index https://flashinfer.ai/whl/cu128 \
--index https://download.pytorch.org/whl/cu128 \
uv pip install "nemo-safe-synthesizer[cu130,engine]" \
--index https://flashinfer.ai/whl/cu130 \
--index https://download.pytorch.org/whl/cu130 \
--index-strategy unsafe-best-match
```

Expand Down
Loading
Loading