Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/_snippets/nvidia-build-model-provider.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
!!! note
The platform pre-configures a `system/nvidia-build` model provider during startup.
`nemo setup` pre-configures a `default/nvidia-build` model provider during local startup.
This provider routes inference requests to models hosted on `build.nvidia.com` using the API base URL `https://integrate.api.nvidia.com`
and the NGC API key with `Public API Endpoints` permissions provided during deployment (automatically saved as the built-in `system/ngc-api-key` secret).
and the NGC API key with `Public API Endpoints` permissions provided during deployment.

You can verify this provider exists by running `nemo inference providers list --workspace system`.
You can verify this provider exists by running `nemo inference providers list --workspace default`.

The tutorials in these docs use this provider for inference, but you can alternatively create your own and use it instead.
1 change: 1 addition & 0 deletions docs/safe-synthesizer/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
tutorials/evaluation_report.html
57 changes: 51 additions & 6 deletions docs/safe-synthesizer/about/host-local-development.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,12 @@
<!-- @nemo-nb: process -->
<!-- @nemo-nb: skip-test -->
<a id="host-local-development"></a>
# Host-Local Development and Testing

Run {{nss_short_name}} on your machine's GPU with `nemo safe-synthesizer run-local`. This page covers the plugin CLI only (`run-local` and `runtime`). It does not cover platform job submission or `nemo safe-synthesizer jobs …` commands (not exposed in the CLI today).
# Local and Subprocess Execution
Comment thread
coderabbitai[bot] marked this conversation as resolved.

Run {{nss_short_name}} on your machine's GPU with `nemo safe-synthesizer run-local`. The public command is a local subprocess wrapper: the main NeMo CLI starts a separate Safe Synthesizer runtime Python, and that runtime executes the synthesis task module.

This page covers local execution only. Platform job submission uses the Jobs API or SDK; the `nemo safe-synthesizer` CLI exposes `run-local` and `runtime`.

## Prerequisites

Expand All @@ -25,7 +28,18 @@ uv run nemo safe-synthesizer --help
# Commands: run-local, runtime
```

## Run a job locally
## Execution modes

There are two local paths:

| Mode | Command | Use it when |
|------|---------|-------------|
| Managed local subprocess | `uv run nemo safe-synthesizer run-local ...` | You want the supported plugin CLI. This creates the parent CLI process, then launches the runtime Python subprocess. |
| Direct local task | `<runtime-python> -m nemo_safe_synthesizer_plugin.tasks.safe_synthesizer run-local ...` | You are debugging the task process itself and want to bypass the parent CLI wrapper. |

Both modes run on the host GPU and write artifacts to the local filesystem. Both accept the same task arguments: `--spec-file`, `--workspace`, `--output-dir`, and optional `--data-source`.

## Run with the managed local subprocess

Use a job spec JSON (example in `plugins/nemo-safe-synthesizer/src/nemo_safe_synthesizer_plugin/nss-job.json`) and a local input file:

Expand All @@ -44,6 +58,39 @@ uv run nemo safe-synthesizer run-local \
| `--output-dir` | Where artifacts are written (default `./nss-output`) |
| `--workspace` | Workspace label for spec fields that reference workspaces (default `default`) |

The parent command launches a subprocess equivalent to:

```bash
<runtime-python> -m nemo_safe_synthesizer_plugin.tasks.safe_synthesizer run-local \
--workspace default \
--spec-file ./nss-job.json \
--data-source ./input.csv \
--output-dir ./nss-output
```

Find the configured runtime Python with:

```bash
uv run nemo safe-synthesizer runtime info
```

## Run the local task directly

Direct task execution is useful when you need to reproduce a subprocess failure without the parent CLI wrapper.

```bash
$(uv run nemo safe-synthesizer runtime info | awk -F': ' '/^python:/ {print $2}') \
-m nemo_safe_synthesizer_plugin.tasks.safe_synthesizer run-local \
--workspace default \
--spec-file ./nss-job.json \
--data-source ./input.csv \
--output-dir ./nss-output
```

If the runtime Python does not exist, run `uv run nemo safe-synthesizer runtime setup` first.

If you omit `--data-source`, the task downloads `data_source` from the platform Files service. Use `--data-source` for offline local files.

### Output layout

| Path | Description |
Expand Down Expand Up @@ -161,7 +208,5 @@ Requires `RUN_NSS_LOCAL_E2E=1`, CUDA, and `nemo safe-synthesizer runtime setup`.

## Related topics

- [Parameters Reference](reference.md) — spec and `config` fields
- [Getting Started](../getting-started.md) — GPU and platform context
- [Jobs](jobs.md) — platform job lifecycle (separate from this run-local guide)
- [Getting Started](../getting-started.md) — GPU and local runtime prerequisites
- Plugin README: `plugins/nemo-safe-synthesizer/README.md`
2 changes: 1 addition & 1 deletion docs/safe-synthesizer/about/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ Get hands-on experience with Safe Synthesizer through step-by-step tutorials.

Understand the job lifecycle, configuration, and execution for Safe Synthesizer pipelines.

- **[Host-Local Development](host-local-development.md)**
- **[Local and Subprocess Execution](host-local-development.md)**
Comment thread
mckornfield marked this conversation as resolved.

---

Expand Down
6 changes: 3 additions & 3 deletions docs/safe-synthesizer/about/jobs.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ When the job completes, access:

For **platform jobs**, set `pretrained_model_job` in the job spec to a completed job that has an **`adapter`** result in Files. Reuse is generation-only (no retraining). Use either `pretrained_model_job` or `config.training.pretrained_model`, not both.

For **host-local** development (`nemo safe-synthesizer run-local`), set `config.training.pretrained_model` to a local adapter or work directory from an earlier run. See [Host-Local Development and Testing](host-local-development.md).
For **host-local** development (`nemo safe-synthesizer run-local`), set `config.training.pretrained_model` to a local adapter or work directory from an earlier run. See [Local and Subprocess Execution](host-local-development.md).

## Job Builder API

Expand All @@ -120,7 +120,7 @@ import os
import pandas as pd

from nemo_platform import NeMoPlatform
from nemo_platform.beta.safe_synthesizer.job_builder import SafeSynthesizerJobBuilder
from nemo_safe_synthesizer_plugin.sdk.job_builder import SafeSynthesizerJobBuilder
Comment thread
mckornfield marked this conversation as resolved.

# Placeholders
df: pd.DataFrame = pd.DataFrame()
Expand Down Expand Up @@ -309,7 +309,7 @@ kubectl get events -n <namespace> --sort-by='.lastTimestamp'

## Related Topics

- [Host-Local Development and Testing](host-local-development.md): `run-local`, adapter reuse, and plugin tests
- [Local and Subprocess Execution](host-local-development.md): `run-local`, adapter reuse, and plugin tests
- [safe-synthesizer-101](../tutorials/safe-synthesizer-101.md): Get started with {{nss_short_name}} jobs
- [index](../tutorials/index.md): More hands-on tutorials
- [reference](reference.md): Full parameter reference
4 changes: 2 additions & 2 deletions docs/safe-synthesizer/about/reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Top-level fields on the Safe Synthesizer job spec (alongside `config`):
| `pretrained_model_job` | Prior completed job whose **`adapter`** result in Files is reused for **generation-only** synthesis. Format: `<job>` or `<workspace>/<job>`. Mutually exclusive with `config.training.pretrained_model`. |
| `hf_token_secret` | Platform secret name for Hugging Face token during model initialization |

For host-local runs, see [Host-Local Development and Testing](host-local-development.md). Reuse a local adapter with `config.training.pretrained_model`, not `pretrained_model_job`.
For host-local runs, see [Local and Subprocess Execution](host-local-development.md). Reuse a local adapter with `config.training.pretrained_model`, not `pretrained_model_job`.

## Top-Level Configuration

Expand Down Expand Up @@ -95,7 +95,7 @@ import os
import pandas as pd

from nemo_platform import NeMoPlatform
from nemo_platform.beta.safe_synthesizer.job_builder import SafeSynthesizerJobBuilder
from nemo_safe_synthesizer_plugin.sdk.job_builder import SafeSynthesizerJobBuilder
Comment thread
mckornfield marked this conversation as resolved.

# Placeholders
df: pd.DataFrame = pd.DataFrame()
Expand Down
19 changes: 10 additions & 9 deletions docs/safe-synthesizer/getting-started.md
Original file line number Diff line number Diff line change
@@ -1,26 +1,26 @@
<a id="nss-getting-started"></a>
# Getting Started with {{nss_short_name}}

Get started with {{nss_short_name}} for generating private synthetic versions of sensitive tabular datasets.
Get started with {{nss_short_name}} for generating private synthetic versions of sensitive tabular datasets on a host GPU.

## Prerequisites

Before using {{nss_short_name}}, complete the [{{platform_name}} Quickstart](../get-started/quickstart.md) to install the CLI/SDK and deploy the platform.
Before using {{nss_short_name}}, complete [Setup](../get-started/setup.md) to install the CLI/SDK.

{{nss_short_name}} has the following additional requirements:

- An NVIDIA GPU **on the host machine** with 80GB+ VRAM (check with `nvidia-smi`). This is separate from any GPU inside a NIM container Safe Synthesizer training runs directly on the host.
- An NVIDIA GPU **on the host machine** with 80GB+ VRAM (check with `nvidia-smi`). This is separate from any GPU inside a NIM container; Safe Synthesizer training runs directly on the host.
- Sufficient disk space for generated datasets (50GB+ recommended)

For general platform troubleshooting (port conflicts, health checks, and so on), refer to the [main quickstart guide](../get-started/quickstart.md).
For general platform troubleshooting (port conflicts, health checks, and so on), refer to [Setup](../get-started/setup.md).

--8<-- "_snippets/nvidia-build-model-provider.md"

---

## Host-local CLI

For GPU development on your machine, install the Safe Synthesizer plugin from this repository and use `nemo safe-synthesizer run-local` (see [Host-Local Development and Testing](about/host-local-development.md)):
For GPU development on your machine, install the Safe Synthesizer plugin from this repository and use `nemo safe-synthesizer run-local` (see [Local and Subprocess Execution](about/host-local-development.md)):

```shell
BOOTSTRAP_LOCAL_PLUGIN_DIRS=plugins/nemo-safe-synthesizer make bootstrap-python
Expand All @@ -31,15 +31,16 @@ uv run nemo safe-synthesizer run-local \
--output-dir ./nss-output
```

Platform job submission (Jobs API, Studio, tutorials) is documented separately in [Jobs](about/jobs.md) and the [tutorials](tutorials/index.md). The `nemo safe-synthesizer` CLI today exposes **run-local** and **runtime** only.
The `run-local` command launches the Safe Synthesizer task in a separate runtime Python subprocess. The `nemo safe-synthesizer` CLI today exposes **run-local** and **runtime** only; platform job submission uses the Jobs API or SDK.

---

## Next Steps

Run one of the [tutorials](tutorials/index.md) to create your first synthetic dataset:
Create your first synthetic dataset:

- [Safe Synthesizer 101 Tutorial](tutorials/safe-synthesizer-101.md) - A beginner-friendly introduction
- [Differential Privacy Tutorial](tutorials/differential-privacy.md) - Generate differentially-private synthetic data
- [Safe Synthesizer 101 Tutorial](tutorials/safe-synthesizer-101.md) - a beginner-friendly introduction
Comment thread
mckornfield marked this conversation as resolved.
- [Local and Subprocess Execution](about/host-local-development.md) - local CLI and runtime task details
- [SDK Resources](sdk-resources.md) - Python SDK methods for jobs, builders, logs, and results

---
2 changes: 1 addition & 1 deletion docs/safe-synthesizer/llms.txt
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
Jobs are created using `SafeSynthesizerJobBuilder` from the Python SDK:

```python
from nemo_platform.beta.safe_synthesizer.job_builder import SafeSynthesizerJobBuilder
from nemo_safe_synthesizer_plugin.sdk.job_builder import SafeSynthesizerJobBuilder

builder = (
SafeSynthesizerJobBuilder(client)
Expand Down
97 changes: 97 additions & 0 deletions docs/safe-synthesizer/sdk-resources.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
<a id="safe-synthesizer-nmp-sdk-resources"></a>
# Safe Synthesizer {{platform_name}} SDK Resources

The `nemo_safe_synthesizer_plugin.sdk` module provides {{platform_name}}-specific helpers for creating and monitoring {{nss_short_name}} jobs. Use these objects when you want to submit jobs through the platform Jobs service and retrieve platform-managed results.

## SafeSynthesizerResource

`SafeSynthesizerResource` is the entry point mounted on a `NeMoPlatform` client:

```python
import os

from nemo_platform import NeMoPlatform

client = NeMoPlatform(
base_url=os.environ.get("NMP_BASE_URL", "http://localhost:8080"),
workspace="default",
)
safe_synthesizer = client.safe_synthesizer
```

An async variant with the same namespace is available as `AsyncNeMoPlatform.safe_synthesizer`.

## SafeSynthesizerJobsResource

`client.safe_synthesizer.jobs` calls the plugin API for {{nss_short_name}} jobs.

| Method | Description |
|--------|-------------|
| `create(*, spec, name=None, workspace=None, project=None, timeout=None, **params)` | Creates a {{nss_short_name}} platform job from a job spec. |
| `list(*, workspace=None, **params)` | Lists {{nss_short_name}} jobs in a workspace. |
| `retrieve(name, *, workspace=None)` | Retrieves one {{nss_short_name}} job by name. |
| `get_status(name, *, workspace=None)` | Returns the job status from the platform Jobs service. |
| `get_logs(name, *, workspace=None, **kwargs)` | Returns paginated job logs from the platform Jobs service. |

The async resource exposes the same methods as `async def` methods.

## SafeSynthesizerJobBuilder

`SafeSynthesizerJobBuilder` assembles a job spec, uploads local datasets to Files, submits the job, and returns a `SafeSynthesizerJob` wrapper.

```python
import pandas as pd

from nemo_safe_synthesizer_plugin.sdk.job_builder import SafeSynthesizerJobBuilder

df = pd.DataFrame({"text": ["sample record"]})

job = (
SafeSynthesizerJobBuilder(client, workspace="default")
.with_data_source(df)
.with_classify_model_provider("default/nvidia-build")
.with_replace_pii()
.synthesize()
.create_job(name="safe-synth-job", project="default-project")
)
```

| Method | Description |
|--------|-------------|
| `with_data_source(data_source)` | Sets a pandas DataFrame or local `.csv`, `.parquet`, `.json`, or `.jsonl` file as input. Local data is uploaded to Files before submission. |
| `with_data(config=None, **kwargs)` | Sets data preparation parameters. |
| `with_train(config=None, **kwargs)` | Sets fine-tuning parameters. |
| `with_generate(config=None, **kwargs)` | Sets generation parameters and enables synthesis. |
| `synthesize()` | Enables synthesis. |
| `with_evaluate(config=None, **kwargs)` | Sets evaluation parameters. |
| `with_differential_privacy(config=None, **kwargs)` | Sets DP-SGD parameters. |
| `with_time_series(config=None, **kwargs)` | Sets time-series parameters. |
| `with_replace_pii(config=None, **kwargs)` | Enables PII replacement. |
| `with_classify_model_provider(provider_name)` | Sets the Inference Gateway provider used for PII column classification. Pair with `with_replace_pii()`. |
| `with_hf_token_secret(secret_name)` | Passes a platform secret name as `HF_TOKEN` to the runtime job. |
| `with_pretrained_model_job(job_name)` | Reuses a prior job's `adapter` result for generation-only synthesis. |
| `resolve_job_config()` | Uploads data and validates the generated job spec without submitting. |
| `create_job(**kwargs)` | Submits the job and returns `SafeSynthesizerJob`. |

## SafeSynthesizerJob

`SafeSynthesizerJob` is a convenience wrapper returned by the builder.

| Method | Description |
|--------|-------------|
| `fetch_status()` | Returns the current platform job status string. |
| `fetch_status_info()` | Returns the full platform job status response. |
| `wait_for_completion(poll_interval=10, verbose=True, log_timeout=None)` | Polls status and logs until the job reaches a terminal state. Raises `RuntimeError` for `error` or `cancelled`. |
| `fetch_logs(timeout=None)` | Iterates over platform job log entries. |
| `print_logs(timeout=None)` | Prints platform job logs to stdout. |
| `fetch_data()` | Downloads the `synthetic-data` result and returns it as a pandas DataFrame. |
| `fetch_summary()` | Downloads the `summary` result as a `SafeSynthesizerSummary`. |
| `fetch_report()` | Downloads the `evaluation-report` result as HTML. |
| `save_report(path)` | Saves the HTML evaluation report to a local file. |
| `display_report_in_notebook(width="100%", height=1000)` | Displays the evaluation report in a notebook. |

## Related Topics

- [Safe Synthesizer 101](tutorials/safe-synthesizer-101.md) - submit and monitor a first job
- [Safe Synthesizer Jobs](about/jobs.md) - understand job lifecycle and troubleshooting
- [Parameters Reference](about/reference.md) - review job spec and configuration fields
4 changes: 2 additions & 2 deletions docs/safe-synthesizer/tutorials/differential-privacy.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ fi
import os
import pandas as pd
from nemo_platform import NeMoPlatform
from nemo_platform.beta.safe_synthesizer.job_builder import SafeSynthesizerJobBuilder
from nemo_safe_synthesizer_plugin.sdk.job_builder import SafeSynthesizerJobBuilder

# Configure client
client = NeMoPlatform(
Expand Down Expand Up @@ -262,7 +262,7 @@ for i, exp in enumerate(experiments):
Configure differential privacy with custom parameters:

```python
from nemo_platform.beta.safe_synthesizer.config import DifferentialPrivacyHyperparams
from nemo_safe_synthesizer_plugin.sdk.config import DifferentialPrivacyHyperparams

# Create custom privacy configuration
privacy_config = DifferentialPrivacyHyperparams(
Expand Down
Loading
Loading