-
Notifications
You must be signed in to change notification settings - Fork 20
feat: add repo-root skills for NVSkills catalog #104
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 3 commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
6ac35cb
feat: mirror plugin and setup skills under repo-root skills/
ngoncharenko fc68896
refactor: symlink catalog skills to plugin sources, drop SETUP.md
ngoncharenko 656a9a3
refactor: keep evaluator and data-designer skills under skills/
ngoncharenko dce6bb6
chore(skills): symlink plugin skill reflections
ngoncharenko f3031fc
docs: avoid secret argv in evaluator auth guide
ngoncharenko 3bbce23
Merge branch 'main' into ngoncharenko/add-root-skills
ngoncharenko File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
6 changes: 3 additions & 3 deletions
6
sdk/python/nemo-platform/src/nemo_platform/skills/inference/SKILL.md
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
2 changes: 1 addition & 1 deletion
2
sdk/python/nemo-platform/src/nemo_platform/skills/nemo-teardown/SKILL.md
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,107 @@ | ||
| --- | ||
| name: data-designer | ||
| description: Use when the user wants to create a dataset, generate synthetic data, or build a data generation pipeline. | ||
| argument-hint: [describe the dataset you want to generate] | ||
| --- | ||
|
|
||
| # Before You Start | ||
|
|
||
| Do not explore the workspace first. The workflow's Learn step gives you everything you need. | ||
|
|
||
| # Goal | ||
|
|
||
| Build a synthetic dataset using the Data Designer library that matches this description: | ||
|
|
||
| $ARGUMENTS | ||
|
|
||
| # Workflow | ||
|
|
||
| Use **Autopilot** mode if the user implies they don't want to answer questions — e.g., they say something like "be opinionated", "you decide", "make reasonable assumptions", "just build it", "surprise me", etc. Otherwise, use **Interactive** mode (default). | ||
|
|
||
| Read **only** the workflow file that matches the selected mode, then follow it: | ||
|
|
||
| - **Interactive** → read `workflows/interactive.md` | ||
| - **Autopilot** → read `workflows/autopilot.md` | ||
|
|
||
| # Rules | ||
|
|
||
| - Keep all columns in the output by default. The only exceptions for dropping a column are: (1) the user explicitly asks, or (2) it is a helper column that exists solely to derive other columns (e.g., a sampled person object used to extract name, city, etc.). When in doubt, keep the column. | ||
| - Do not suggest or ask about seed datasets. Only use one when the user explicitly provides seed data or asks to build from existing records. When using a seed, read `references/seed-datasets.md`. | ||
| - When the dataset requires person data (names, demographics, addresses), read `references/person-sampling.md`. | ||
| - If a dataset script that matches the dataset description already exists, ask the user whether to edit it or create a new one. | ||
| - For commands and context specific to this NeMo Platform plugin (e.g., sourcing model configs from IGW providers or in-script `ModelConfig`s, installing or publishing Nemotron Personas locales, platform-side resource pointers), read `references/nemo-platform-plugin-additions.md`. | ||
|
|
||
| # Usage Tips and Common Pitfalls | ||
|
|
||
| - **Sampler and validation columns need both a type and params.** E.g., `sampler_type="category"` with `params=dd.CategorySamplerParams(...)`. | ||
| - **Jinja2 templates** in `prompt`, `system_prompt`, and `expr` fields: reference columns with `{{ column_name }}`, nested fields with `{{ column_name.field }}`. | ||
| - **`SamplerColumnConfig`:** Takes `params`, not `sampler_params`. | ||
| - **LLM judge score access:** `LLMJudgeColumnConfig` produces a nested dict where each score name maps to `{reasoning: str, score: int}`. To get the numeric score, use the `.score` attribute. For example, for a judge column named `quality` with a score named `correctness`, use `{{ quality.correctness.score }}`. Using `{{ quality.correctness }}` returns the full dict, not the numeric score. | ||
|
|
||
| # Troubleshooting | ||
|
|
||
| - **`nemo data-designer` CLI not found:** Tell the user that `nemo data-designer` is not installed in this environment (requires Python >= 3.11). Ask if they would like you to create a virtual environment and install it, or if they prefer to do it themselves. Do not install anything without the user's permission. | ||
| - **Network errors during preview:** A sandbox environment may be blocking outbound requests. Ask the user for permission to retry the command with the sandbox disabled. Only as a last resort, if retrying outside the sandbox also fails, tell the user to run the command themselves. | ||
|
|
||
| # Output Template | ||
|
|
||
| Write a Python file to the current directory with a `load_config_builder()` function returning a `DataDesignerConfigBuilder`. Name the file descriptively (e.g., `customer_reviews.py`). Use PEP 723 inline metadata for dependencies. | ||
|
|
||
| ```python | ||
| # /// script | ||
| # dependencies = [ | ||
| # "data-designer", # always required | ||
| # "pydantic", # only if this script imports from pydantic | ||
| # # add additional dependencies here | ||
| # ] | ||
| # /// | ||
| import data_designer.config as dd | ||
| from pydantic import BaseModel, Field | ||
|
|
||
|
|
||
| # Use Pydantic models when the output needs to conform to a specific schema | ||
| class MyStructuredOutput(BaseModel): | ||
| field_one: str = Field(description="...") | ||
| field_two: int = Field(description="...") | ||
|
|
||
|
|
||
| # Use custom generators when built-in column types aren't enough | ||
| @dd.custom_column_generator( | ||
| required_columns=["col_a"], | ||
| side_effect_columns=["extra_col"], | ||
| ) | ||
| def generator_function(row: dict) -> dict: | ||
| # add custom logic here that depends on "col_a" and update row in place | ||
| row["name_in_custom_column_config"] = "custom value" | ||
| row["extra_col"] = "extra value" | ||
| return row | ||
|
|
||
|
|
||
| def load_config_builder() -> dd.DataDesignerConfigBuilder: | ||
| config_builder = dd.DataDesignerConfigBuilder( | ||
| # Declaring model configs programmatically here is the portable path: | ||
| # it works for both local `run` and cluster `submit`, while the local | ||
| # YAML registry alternative only works for `run`. The provider below | ||
| # is a common default created during `nemo setup` — confirm it (or | ||
| # discover others) with `nemo inference providers list`. See | ||
| # references/nemo-platform-plugin-additions.md for the local-YAML alternative. | ||
| model_configs=[ | ||
| dd.ModelConfig( | ||
| alias="text", | ||
| model="...", | ||
| provider="default/nvidia-build", | ||
| inference_parameters=dd.ChatCompletionInferenceParams(), | ||
| ), | ||
| ], | ||
| ) | ||
|
|
||
| # Seed dataset (only if the user explicitly mentions a seed dataset path) | ||
| # config_builder.with_seed_dataset(dd.LocalFileSeedSource(path="path/to/seed.parquet")) | ||
|
|
||
| # config_builder.add_column(...) | ||
| # config_builder.add_processor(...) | ||
|
|
||
| return config_builder | ||
| ``` | ||
|
|
||
| Only include Pydantic models, custom generators, seed datasets, and extra dependencies when the task requires them. Prefer including `model_configs` when the dataset uses LLM columns — declaring it in the script keeps the config portable between local `run` and cluster `submit`, while the local YAML registry alternative only works for `run`. |
91 changes: 91 additions & 0 deletions
91
skills/nemo-data-designer-plugin/references/nemo-platform-plugin-additions.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,91 @@ | ||
| # NeMo Plugin Additions | ||
|
|
||
| This skill ships in the NeMo Platform data-designer plugin. The CLI surface is `nemo data-designer …`. Most subcommands accept the same arguments as the upstream `data-designer` CLI; the differences are documented below. | ||
|
|
||
| ## `preview` and `create`: local vs cluster | ||
|
|
||
| Upstream's `preview` and `create` are flat commands that take the config path positional directly. In the plugin, both are command groups with two execution modes: | ||
|
|
||
| - `nemo data-designer preview run <path> [flags]` — local in-process execution. Use this in the standard skill workflow. | ||
| - `nemo data-designer preview submit <path> [flags]` — submit to a NeMo Platform cluster over HTTP. Use only when the user explicitly asks for cluster execution. | ||
| - `nemo data-designer create run <path> [flags]` — local in-process generation. | ||
| - `nemo data-designer create submit <path> [flags]` — cluster submission (also supports `--profile <profile>`). | ||
|
|
||
| Default to `run` for the iterative preview-and-iterate workflow; reach for `submit` only when the user calls it out (e.g., "submit on the cluster", "run this on the platform"). The args after `run` / `submit` match upstream's `preview` / `create` args. | ||
|
|
||
| ## Model configs | ||
|
|
||
| The upstream skill assumes model aliases come from a YAML registry under `~/.data-designer/`, populated via `nemo data-designer config models` / `nemo data-designer config providers`. In this plugin you have two additional sources, and either is a first-class option — `agent context` does **not** see them. | ||
|
|
||
| **Declare `ModelConfig`s programmatically in the script.** `DataDesignerConfigBuilder` accepts model configs directly, either via its constructor or `.add_model_config(...)`: | ||
|
|
||
| ```python | ||
| import data_designer.config as dd | ||
|
|
||
| def load_config_builder() -> dd.DataDesignerConfigBuilder: | ||
| config_builder = dd.DataDesignerConfigBuilder( | ||
| model_configs=[ | ||
| dd.ModelConfig( | ||
| alias="text", | ||
| model="...", | ||
| provider="default/nvidia-build", | ||
| inference_parameters=dd.ChatCompletionInferenceParams(), | ||
| ), | ||
| ], | ||
| ) | ||
| ... | ||
| ``` | ||
|
|
||
| Pick the right `inference_parameters` class for the generation type: `ChatCompletionInferenceParams`, `EmbeddingInferenceParams`, or `ImageInferenceParams`. The class determines the alias's `generation_type` and which column types can use it. | ||
|
|
||
| **Reference an Inference Gateway-managed model provider.** `ModelConfig.provider` may be a bare provider name (resolved in the active workspace) or `<workspace>/<provider>`. The plugin's request handler resolves local providers first, then falls back to looking up the name via the Inference Gateway, so the same `ModelConfig` works whether the user has local providers configured or not. | ||
|
|
||
| Discover available Inference Gateway providers with `nemo inference providers list`. A common default created during `nemo setup` is `default/nvidia-build`, but it's optional — confirm before relying on it. If the user mentions a provider by name (e.g., "use my-vllm"), trust the name and let the registry surface a clear error at preview time if it isn't reachable. | ||
|
|
||
| **Default to programmatic declaration with an Inference Gateway provider.** It's the portable path: declaring `model_configs` in the script works for both local `run` and cluster `submit`, whereas relying on the local YAML registry only works for `run`. Most plugin workflows iterate locally before submitting, so the portable path saves a rewrite later. | ||
|
|
||
| When using an Inference Gateway provider, the `model` field in the `dd.ModelConfig` should use the `served_model_name` as understood by Inference Gateway, not the `model_entity_id`. | ||
|
|
||
| If `agent context` shows no usable aliases, that is **not** a blocker — it only means the local YAML registry is unconfigured. Fall back to local YAML aliases only when the user has explicitly configured them and asks for that path. | ||
|
|
||
| ## Personas | ||
|
|
||
| The plugin adds a `personas` command group on top of upstream Data Designer. Use it to install Nemotron Personas locales locally and to publish them as NeMo Platform filesets so cluster-side jobs can read them. | ||
|
|
||
| **Install one or more locales locally**: | ||
|
|
||
| ```bash | ||
| # List available locales and their sizes | ||
| nemo data-designer personas download --list | ||
|
|
||
| # Interactive selection | ||
| nemo data-designer personas download | ||
|
|
||
| # Specific locales | ||
| nemo data-designer personas download --locale en_US --locale ja_JP | ||
|
|
||
| # All locales | ||
| nemo data-designer personas download --all | ||
| ``` | ||
|
|
||
| Locales download to `~/.data-designer/managed-assets/datasets/`, which is also the path the `"person"` sampler reads from. After installing, `references/person-sampling.md` covers the general column-usage flow without modification — the plugin doesn't change how persona columns work. | ||
|
|
||
| **Publish a locale as a NeMo Platform fileset** (so NeMo Platform-side jobs that need persona data can read it): | ||
|
|
||
| ```bash | ||
| nemo data-designer personas make-fileset \ | ||
| --locale en_US \ | ||
| --api-key-secret <workspace>/<secret-name> | ||
| ``` | ||
|
|
||
| Requires an NGC API key secret already registered in NeMo Platform. To create the secret in the same call, add `--api-key-env-var <ENV_VAR>` and set that env var to the API key value before running. | ||
|
|
||
| ## Related NeMo Platform commands | ||
|
|
||
| When the user already has NeMo Platform-side resources configured, prefer pointing them at those rather than the local Data Designer config: | ||
|
|
||
| - `nemo inference providers list` / `nemo models list` — NeMo Platform-side inference providers and models. | ||
| - `nemo secrets` — manage API keys used by `personas make-fileset` and other NeMo Platform-side flows. | ||
| - `nemo files` — manage filesets, including persona filesets created above. | ||
|
|
||
| These are alternatives to the local `~/.data-designer/` configuration the upstream skill assumes; both work, and which to use depends on whether the user is iterating locally or running on a cluster. |
46 changes: 46 additions & 0 deletions
46
skills/nemo-data-designer-plugin/references/person-sampling.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| # Person Sampling Reference | ||
|
|
||
| ## Sampler types | ||
|
|
||
| Prefer `"person"` when the locale is downloaded — it provides census-grounded demographics and optional personality traits. Fall back to `"person_from_faker"` when the locale isn't available. | ||
|
|
||
|
|
||
| | `sampler_type` | Params class | When to use | | ||
| | --------------------- | ------------------------------ | --------------------------------------------------------------------------------------------------- | | ||
| | `"person"` | `PersonSamplerParams` | **Preferred.** Locale downloaded to `~/.data-designer/managed-assets/datasets/` by default. | | ||
| | `"person_from_faker"` | `PersonFromFakerSamplerParams` | Fallback when locale not downloaded. Basic names/addresses via Faker, not demographically accurate. | | ||
|
|
||
|
|
||
| ## Usage | ||
|
|
||
| The sampled person column is a nested dict. You can keep it as-is in the final dataset, or set `drop=True` to remove it and extract only the fields you need via `ExpressionColumnConfig`: | ||
|
|
||
| ```python | ||
| # Keep the full person dict in the output | ||
| config_builder.add_column(dd.SamplerColumnConfig( | ||
| name="person", sampler_type="person", | ||
| params=dd.PersonSamplerParams(locale="en_US"), | ||
| )) | ||
|
|
||
| # Or drop it and extract specific fields | ||
| config_builder.add_column(dd.SamplerColumnConfig( | ||
| name="person", sampler_type="person", | ||
| params=dd.PersonSamplerParams(locale="en_US"), drop=True, | ||
| )) | ||
| config_builder.add_column(dd.ExpressionColumnConfig( | ||
| name="full_name", | ||
| expr="{{ person.first_name }} {{ person.last_name }}", dtype="str", | ||
| )) | ||
| ``` | ||
|
|
||
| Set `with_synthetic_personas=True` when the dataset benefits from personality traits, interests, cultural background, or detailed persona descriptions (e.g., for realistic user simulation or persona-driven prompting). This option is only available with `"person"` — `"person_from_faker"` does not support it. | ||
|
|
||
| ## Person Object Schema | ||
|
|
||
| Fields vary by locale. Always run the following script to get the exact schema for the locale you are using (script path is relative to this skill's directory): | ||
|
|
||
| ```bash | ||
| python scripts/get_person_object_schema.py <locale> | ||
| ``` | ||
|
|
||
| This prints the PII fields (always included) and synthetic persona fields (only included when `with_synthetic_personas=True`) available for that locale. | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.