Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
528 changes: 528 additions & 0 deletions docs/customizer/tutorials/dpo-customization-job.ipynb

Large diffs are not rendered by default.

9 changes: 9 additions & 0 deletions docs/customizer/tutorials/dpo-customization-job.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
title: "DPO Customization"
description: ""
---

<NotebookViewer
name="dpo-customization-job"
colabUrl="https://colab.research.google.com/github/NVIDIA-NeMo/nemo-platform/blob/main/docs/customizer/tutorials/dpo-customization-job.ipynb"
/>
2 changes: 2 additions & 0 deletions docs/fern/components/NotebookViewer.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
import type { ReactNode } from "react";

import distillationCustomizationJob from "./notebooks/distillation-customization-job";
import dpoCustomizationJob from "./notebooks/dpo-customization-job";
import embeddingCustomizationJob from "./notebooks/embedding-customization-job";
import loraCustomizationJob from "./notebooks/lora-customization-job";
import optimizeThroughput from "./notebooks/optimize-throughput";
Expand All @@ -17,6 +18,7 @@ import toolCalling from "./notebooks/tool-calling";
// pages look it up by name: `<NotebookViewer name="sft-customization-job" />`.
const notebooks: Record<string, unknown> = {
"distillation-customization-job": distillationCustomizationJob,
"dpo-customization-job": dpoCustomizationJob,
"embedding-customization-job": embeddingCustomizationJob,
"lora-customization-job": loraCustomizationJob,
"optimize-throughput": optimizeThroughput,
Expand Down
155 changes: 155 additions & 0 deletions docs/fern/components/notebooks/dpo-customization-job.json

Large diffs are not rendered by default.

159 changes: 159 additions & 0 deletions docs/fern/components/notebooks/dpo-customization-job.ts

Large diffs are not rendered by default.

4 changes: 4 additions & 0 deletions docs/fern/scripts/validate-notebook-viewer.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,10 @@ const SOURCE_NOTEBOOKS = {
ROOT,
"../../customizer/tutorials/distillation-customization-job.ipynb",
),
"dpo-customization-job": join(
ROOT,
"../../customizer/tutorials/dpo-customization-job.ipynb",
),
"embedding-customization-job": join(
ROOT,
"../../customizer/tutorials/embedding-customization-job.ipynb",
Expand Down
1 change: 1 addition & 0 deletions docs/troubleshooting/customizer.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ description: ""
- The platform's shared persistent volume is likely full. Budget against the downloaded base checkpoint size: approximately 3× for Full SFT and 1.5× for LoRA. For example, a 70B BF16 checkpoint is approximately 140 GB, so a Full SFT job can require approximately 420 GB of free disk at peak.
- These peak estimates include the base checkpoint and job artifacts; the final Full SFT output itself is one full checkpoint. If you also retain a deployment copy, include it separately in capacity planning.
- Clean up completed job artifacts or increase the PVC size (default: 200Gi at `/var/run/scratch/job`).
- DPO/GRPO jobs also consume ephemeral node storage under `/tmp` via Ray workers — check node disk in addition to the PVC.
- See [ft-tut-understand-models](/documentation/customizer-reference/tutorials/understanding-models-and-training) for full storage requirement details.

**Job fails with OOM (Out of Memory):**
Expand Down
2 changes: 1 addition & 1 deletion plugins/nemo-customizer/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# nemo-customizer

Router service for `/apis/customization`. Training backends (Automodel, Unsloth, …) register as **`nemo.customization.contributors`** entry points (discovered via `nemo_platform_plugin.discovery`).
Router service for `/apis/customization`. Training backends (Automodel, RL, Megatron, …) register as **`nemo.customization.contributors`** entry points (discovered via `nemo_platform_plugin.discovery`).

Registers **`nemo.sdk`** → `customization` for `client.customization.*` (composes contributor SDK modules such as `client.customization.automodel.jobs`).

Expand Down
Loading
Loading