Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 1 addition & 3 deletions docs/anonymizer/quickstart.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,6 @@
title: "Quick Start"
description: ""
---
import SnippetNvidiaBuildModelProvider from "@/snippets/_snippets/nvidia-build-model-provider.mdx";

<a id="anonymizer-quickstart"></a>

This guide walks through previewing and running an Anonymizer job on NeMo Platform.
Expand Down Expand Up @@ -43,7 +41,7 @@ Anonymizer routes inference through the [Inference Gateway service](../run-infer

`nemo setup` walks you through creating a provider secret and registering an Inference Gateway provider as part of the install flow. If you skipped that step or want to add another provider, re-run `nemo setup` — see the [Setup guide](../get-started/setup.md) for details.

<SnippetNvidiaBuildModelProvider />
<Markdown src="/snippets/_snippets/nvidia-build-model-provider.mdx" />
## Step 4: Upload an Input Fileset

Create a small CSV containing PII and upload it to a fileset:
Expand Down
1 change: 0 additions & 1 deletion docs/auditor/index.mdx
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
---
description: Scan and audit large language models for jailbreaks, prompt injection, encoding bypasses, and other safety failures using {{__auditor_short_name}}, powered by garak.
---
import { Button } from "@/components/Button";

<a id="about-auditor"></a>

Expand Down
4 changes: 1 addition & 3 deletions docs/cli/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,6 @@
title: "NeMo Platform CLI"
description: ""
---
import SnippetCliSummary from "@/snippets/_snippets/cli-summary.mdx";

<a id="nemo-ms-cli"></a>

The NeMo Platform CLI (`nemo`) is a command-line tool for interacting with NeMo Platform. It provides a unified interface for managing models, running jobs, deploying inference endpoints, and working with a local setup.
Expand Down Expand Up @@ -79,7 +77,7 @@ The CLI follows a consistent pattern:
nemo [GLOBAL OPTIONS] <command> [<subcommand>...] [OPTIONS]
```

<SnippetCliSummary />
<Markdown src="/snippets/_snippets/cli-summary.mdx" />
## Common Workflows

### Exploring Commands
Expand Down
5 changes: 1 addition & 4 deletions docs/customizer/tutorials/distillation-customization-job.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,7 @@ title: "Knowledge Distillation Customization"
description: ""
---

import { NotebookViewer } from "@/components/NotebookViewer";
import notebook from "@/components/notebooks/distillation-customization-job";

<NotebookViewer
notebook={notebook}
name="distillation-customization-job"
colabUrl="https://colab.research.google.com/github/NVIDIA-NeMo/nemo-platform/blob/main/docs/customizer/tutorials/distillation-customization-job.ipynb"
/>
5 changes: 1 addition & 4 deletions docs/customizer/tutorials/dpo-customization-job.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,7 @@ title: "DPO Customization"
description: ""
---

import { NotebookViewer } from "@/components/NotebookViewer";
import notebook from "@/components/notebooks/dpo-customization-job";

<NotebookViewer
notebook={notebook}
name="dpo-customization-job"
colabUrl="https://colab.research.google.com/github/NVIDIA-NeMo/nemo-platform/blob/main/docs/customizer/tutorials/dpo-customization-job.ipynb"
/>
5 changes: 1 addition & 4 deletions docs/customizer/tutorials/embedding-customization-job.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,7 @@ title: "Embedding Model Customization"
description: ""
---

import { NotebookViewer } from "@/components/NotebookViewer";
import notebook from "@/components/notebooks/embedding-customization-job";

<NotebookViewer
notebook={notebook}
name="embedding-customization-job"
colabUrl="https://colab.research.google.com/github/NVIDIA-NeMo/nemo-platform/blob/main/docs/customizer/tutorials/embedding-customization-job.ipynb"
/>
4 changes: 1 addition & 3 deletions docs/customizer/tutorials/format-training-dataset.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,6 @@
title: "Format Training Dataset"
description: ""
---
import SnippetPrereqs from "@/snippets/_snippets/tutorials/prereqs.mdx";

<a id="fine-tune-format-training-dataset"></a>

Learn how to format a training dataset to work with the model type you want to train, such as a **chat** or **completion** model.
Expand All @@ -15,7 +13,7 @@ Customizer expects _all_ datasets to use JSONL format, where each line in the da

## Prerequisites

<SnippetPrereqs />
<Markdown src="/snippets/_snippets/tutorials/prereqs.mdx" />
<a id="dataset-best-practices"></a>
## Dataset Best Practices

Expand Down
4 changes: 1 addition & 3 deletions docs/customizer/tutorials/import-hf-model.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,13 @@
title: "Import and Fine-Tune Private HuggingFace Models"
description: ""
---
import SnippetPrereqs from "@/snippets/_snippets/tutorials/prereqs.mdx";

<a id="ft-tut-private-reg"></a>

Use this tutorial to learn how to import a private HuggingFace model into NeMo Customizer, fine-tune it with LoRA, and deploy it for inference.

## Prerequisites

<SnippetPrereqs />
<Markdown src="/snippets/_snippets/tutorials/prereqs.mdx" />
### Tutorial-Specific Prerequisites

- Access to Data Store and Deployment Manager service
Expand Down
5 changes: 1 addition & 4 deletions docs/customizer/tutorials/lora-customization-job.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,7 @@ title: "LoRA Model Customization"
description: ""
---

import { NotebookViewer } from "@/components/NotebookViewer";
import notebook from "@/components/notebooks/lora-customization-job";

<NotebookViewer
notebook={notebook}
name="lora-customization-job"
colabUrl="https://colab.research.google.com/github/NVIDIA-NeMo/nemo-platform/blob/main/docs/customizer/tutorials/lora-customization-job.ipynb"
/>
7 changes: 2 additions & 5 deletions docs/customizer/tutorials/metrics.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,6 @@
title: "Checking Your Customization Job Metrics"
description: ""
---
import SnippetPrereqs from "@/snippets/_snippets/tutorials/prereqs.mdx";
import SnippetCustomizerPrereqs from "@/snippets/customizer/tutorials/_snippets/customizer-prereqs.mdx";

<a id="fine-tune-metrics"></a>

After completing a customization job, you can monitor its performance through training and validation metrics. You can access these metrics in three ways:
Expand All @@ -21,8 +18,8 @@ The time to complete this tutorial is approximately 10 minutes.

## Prerequisites

<SnippetPrereqs />
<SnippetCustomizerPrereqs />
<Markdown src="/snippets/_snippets/tutorials/prereqs.mdx" />
<Markdown src="/snippets/customizer/tutorials/_snippets/customizer-prereqs.mdx" />
### Tutorial-Specific Prerequisites

- Completed customization job with a valid ID
Expand Down
5 changes: 1 addition & 4 deletions docs/customizer/tutorials/optimize-throughput.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,7 @@ title: "Optimize for Tokens/GPU Throughput"
description: ""
---

import { NotebookViewer } from "@/components/NotebookViewer";
import notebook from "@/components/notebooks/optimize-throughput";

<NotebookViewer
notebook={notebook}
name="optimize-throughput"
colabUrl="https://colab.research.google.com/github/NVIDIA-NeMo/nemo-platform/blob/main/docs/customizer/tutorials/optimize-throughput.ipynb"
/>
5 changes: 1 addition & 4 deletions docs/customizer/tutorials/sft-customization-job.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,7 @@ title: "Full SFT Customization"
description: ""
---

import { NotebookViewer } from "@/components/NotebookViewer";
import notebook from "@/components/notebooks/sft-customization-job";

<NotebookViewer
notebook={notebook}
name="sft-customization-job"
colabUrl="https://colab.research.google.com/github/NVIDIA-NeMo/nemo-platform/blob/main/docs/customizer/tutorials/sft-customization-job.ipynb"
/>
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,6 @@
title: "Understanding NeMo Customizer: Models, Training, and Resources"
description: ""
---
import SnippetPrereqs from "@/snippets/_snippets/tutorials/prereqs.mdx";
import SnippetCustomizerPrereqs from "@/snippets/customizer/tutorials/_snippets/customizer-prereqs.mdx";

<a id="ft-tut-understand-models"></a>

Learn the fundamentals of how NeMo Customizer works to make informed decisions about your fine-tuning projects. This tutorial covers how models are organized, how adapters attach to base models, training types and GPU requirements, and how to choose the right approach for your use case.
Expand All @@ -19,8 +16,8 @@ The time to complete this tutorial is approximately 15 minutes.
This tutorial focuses on understanding and discovery—no actual training jobs are created.
## Prerequisites

<SnippetPrereqs />
<SnippetCustomizerPrereqs />
<Markdown src="/snippets/_snippets/tutorials/prereqs.mdx" />
<Markdown src="/snippets/customizer/tutorials/_snippets/customizer-prereqs.mdx" />
---

## Core Concepts
Expand Down
7 changes: 2 additions & 5 deletions docs/data-designer/tutorials/basics.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,6 @@
title: "The Basics"
description: ""
---
import SnippetPreviewResults from "@/snippets/data-designer/_snippets/preview-results.mdx";
import SnippetJobResults from "@/snippets/data-designer/_snippets/job-results.mdx";

<a id="data-designer-tutorials-basics"></a>

This tutorial demonstrates the fundamentals of Data Designer by generating a product review dataset.
Expand Down Expand Up @@ -261,7 +258,7 @@ print(df.head())
preview.analysis.to_report()
```

<SnippetPreviewResults />
<Markdown src="/snippets/data-designer/_snippets/preview-results.mdx" />
**Iterate:** Adjust column configurations, prompts, or parameters in your `config_builder`, then run `preview` again until you're satisfied with the results.

### Scaling Up with Jobs
Expand All @@ -287,7 +284,7 @@ analysis = results.load_analysis()
analysis.to_report()
```

<SnippetJobResults />
<Markdown src="/snippets/data-designer/_snippets/job-results.mdx" />
## What Happens Under the Hood

When you use CLI `run`:
Expand Down
7 changes: 2 additions & 5 deletions docs/data-designer/tutorials/seeding.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,6 @@
title: "Seeding with External Datasets"
description: ""
---
import SnippetPreviewResults from "@/snippets/data-designer/_snippets/preview-results.mdx";
import SnippetJobResults from "@/snippets/data-designer/_snippets/job-results.mdx";

<a id="data-designer-tutorials-seeding"></a>

This tutorial demonstrates how to use external datasets as seed data for synthetic data generation in Data Designer.
Expand Down Expand Up @@ -308,7 +305,7 @@ print(df.head())
preview.analysis.to_report()
```

<SnippetPreviewResults />
<Markdown src="/snippets/data-designer/_snippets/preview-results.mdx" />
### Generating the Full Dataset

When you're satisfied with the preview, submit a larger generation job:
Expand All @@ -332,7 +329,7 @@ analysis = results.load_analysis()
analysis.to_report()
```

<SnippetJobResults />
<Markdown src="/snippets/data-designer/_snippets/job-results.mdx" />
## How Seeding Works

When you configure a seed dataset:
Expand Down
4 changes: 1 addition & 3 deletions docs/evaluator/benchmarks/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,6 @@
title: "Evaluation Benchmarks"
description: ""
---
import SnippetDiscoverIndustryBenchmarks from "@/snippets/evaluator/benchmarks/discover-industry-benchmarks.mdx";

<a id="eval-benchmarks-index"></a>

A benchmark is a reusable evaluation suite: one or more metrics paired with a dataset. Instead of redefining metrics and data inputs for every run, you define the benchmark once and run it repeatedly.
Expand All @@ -29,7 +27,7 @@ Custom benchmarks are valuable for domain-specific evaluation where standard ben
| [**Industry Benchmarks**](industry.md) | Compare against published baselines, regression testing, model selection | Canonical datasets (fixed) | Standardized metrics |
| [**Custom Benchmarks**](custom.md) | Domain-specific evaluation, production monitoring, task-specific assessment | Your evaluation data | Your choice of metrics |

<SnippetDiscoverIndustryBenchmarks />
<Markdown src="/snippets/evaluator/benchmarks/discover-industry-benchmarks.mdx" />
## Create Custom Benchmarks

Create a custom benchmark by combining metrics with your dataset. Before creating a benchmark, you will need to [create the metrics](../metrics/index.md) that define how to score your model's outputs.
Expand Down
9 changes: 3 additions & 6 deletions docs/evaluator/benchmarks/industry.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,6 @@
title: "Industry Benchmarks"
description: ""
---
import SnippetDiscoverIndustryBenchmarks from "@/snippets/evaluator/benchmarks/discover-industry-benchmarks.mdx";
import SnippetHfSecret from "@/snippets/evaluator/benchmarks/hf-secret.mdx";

<a id="eval-benchmarks-industry"></a>

## Evaluate with Published Datasets
Expand All @@ -19,7 +16,7 @@ Refer to the [Run an LLM Judge Evaluation](../tutorials/run-llm-judge-evaluation
- **Reproducible Metrics**: Use established methodologies to calculate metrics.
- **Community Standards**: You can compare results across different models and research groups.

<SnippetDiscoverIndustryBenchmarks />
<Markdown src="/snippets/evaluator/benchmarks/discover-industry-benchmarks.mdx" />
### Choosing a Benchmark Variant

Many benchmarks offer multiple variants optimized for different model types:
Expand Down Expand Up @@ -268,7 +265,7 @@ job = client.evaluation.benchmark_jobs.create(
</Tabs>
<Note>

<SnippetHfSecret />
<Markdown src="/snippets/evaluator/benchmarks/hf-secret.mdx" />

</Note>
### Results
Expand Down Expand Up @@ -1147,7 +1144,7 @@ Create an evaluation job with a benchmark that satisfy the required and optional

For benchmarks that require a Hugging Face token or other API keys for external services, create the secret to be referenced by the job.

<SnippetHfSecret />
<Markdown src="/snippets/evaluator/benchmarks/hf-secret.mdx" />

</Note>
```python
Expand Down
1 change: 0 additions & 1 deletion docs/evaluator/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@
title: "About Evaluating"
description: ""
---
import { Button } from "@/components/Button";

<a id="nemo-ms-evaluator-about"></a>

Expand Down
5 changes: 1 addition & 4 deletions docs/example-applications/tool-calling.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,7 @@ title: "Tool-Calling Fine-Tuning with Synthetic Data"
description: ""
---

import { NotebookViewer } from "@/components/NotebookViewer";
import notebook from "@/components/notebooks/tool-calling";

<NotebookViewer
notebook={notebook}
name="tool-calling"
colabUrl="https://colab.research.google.com/github/NVIDIA-NeMo/nemo-platform/blob/main/docs/example-applications/tool-calling.ipynb"
/>
37 changes: 0 additions & 37 deletions docs/fern/assets/NVIDIA_dark.svg

This file was deleted.

36 changes: 0 additions & 36 deletions docs/fern/assets/NVIDIA_light.svg

This file was deleted.

Loading
Loading