Add relevant 26.02 docs to r1.1.0 by sarahyurick · Pull Request #1493 · NVIDIA-NeMo/Curator

sarahyurick · 2026-02-11T22:45:47Z

Notes:

We do not include [benchmark] Add Video Benchmarks #1430 in r1.1.0

r1.1.0 uses output-clip-path
main uses output-path

We do not include Benchmarking script for image pipeline #1441 in r1.1.0

r1.1.0 uses batch_size
main uses dali_batch_size

Signed-off-by: Sarah Yurick <sarahyurick@gmail.com>

copy-pr-bot · 2026-02-11T22:45:51Z

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

Signed-off-by: Sarah Yurick <sarahyurick@gmail.com>

greptile-apps · 2026-02-12T00:11:43Z

Greptile Overview

Greptile Summary

Backports 26.02 release documentation from main to r1.1.0 branch. Updates version identifiers (25.09 → 26.02) across configuration files and adds comprehensive new documentation for synthetic data generation (Nemotron-CC pipelines, LLM client configuration, multilingual Q&A). Enhances existing documentation with improved examples, clarifications, and structural reorganization.

Major additions:

New synthetic data generation guides (docs/curate-text/synthetic/*) covering LLM clients, multilingual Q&A, and advanced Nemotron-CC pipelines
Enhanced video deduplication documentation with SemanticDeduplicationWorkflow examples
Expanded installation guide with FFmpeg setup, CUDA 12 requirements, and container recommendations
Updated release notes restructured for 26.02 release

Key improvements:

Reorganized table of contents (Setup & Deployment moved before content sections)
Added broken link false positive for HuggingFace token URL
Expanded heuristic filter and distributed classifier documentation
Improved tutorial README with command-line references and usage examples

The PR description explicitly notes intentional version-specific exclusions (#1430 uses output-clip-path in r1.1.0 vs output-path in main; #1441 uses batch_size vs dali_batch_size), ensuring proper compatibility with the r1.1.0 codebase.

Confidence Score: 5/5

This PR is safe to merge with no risk - it contains only documentation updates with no code changes
Documentation-only PR with 44 markdown/config files updated. Changes are well-scoped backports from main to r1.1.0 branch with clear version management (25.09 → 26.02). The PR author has documented intentional exclusions to maintain compatibility with r1.1.0 codebase. No code logic changes, no security concerns, and all updates follow established documentation patterns.
No files require special attention

Important Files Changed

Filename	Overview
docs/about/release-notes/index.md	Updates release notes from 25.09 to 26.02, restructuring content and adding new feature documentation
docs/conf.py	Updates release version from 25.09 to 26.02 and container version string
docs/versions1.json	Adds 26.02 version as preferred release in version dropdown configuration
docs/curate-text/synthetic/index.md	New comprehensive synthetic data generation guide with overview and getting started sections
docs/curate-text/synthetic/llm-client.md	New documentation for LLM client configuration, covering AsyncOpenAIClient setup and performance tuning
docs/curate-text/synthetic/nemotron-cc/index.md	New advanced documentation for Nemotron-CC pipelines with architecture diagrams and composable patterns
docs/curate-text/synthetic/nemotron-cc/tasks.md	New comprehensive task reference for Nemotron-CC with examples for each task type
docs/admin/installation.md	Updates installation guide with CUDA 12 requirements, FFmpeg setup, and container recommendations
docs/curate-video/process-data/dedup.md	Major update adding SemanticDeduplicationWorkflow with improved examples and documentation structure
tutorials/synthetic/README.md	Expands synthetic tutorial README with Nemotron-CC examples, command-line reference, and usage guides
docs/index.md	Reorganizes table of contents, moving Setup & Deployment before content sections and adding Synthetic Data
docs/curate-text/process-data/quality-assessment/heuristic.md	Major expansion of heuristic filter documentation with examples and configuration details

greptile-apps

_{44 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}

greptile-apps

_{44 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}

* add release notes Signed-off-by: Sarah Yurick <sarahyurick@gmail.com> * add more pages Signed-off-by: Sarah Yurick <sarahyurick@gmail.com> * add more pages Signed-off-by: Sarah Yurick <sarahyurick@gmail.com> * add new sdg docs Signed-off-by: Sarah Yurick <sarahyurick@gmail.com> * update remaining files from sdg docs Signed-off-by: Sarah Yurick <sarahyurick@gmail.com> * continue adding more changes Signed-off-by: Sarah Yurick <sarahyurick@gmail.com> * more video docs Signed-off-by: Sarah Yurick <sarahyurick@gmail.com> * add remaining updates Signed-off-by: Sarah Yurick <sarahyurick@gmail.com> --------- Signed-off-by: Sarah Yurick <sarahyurick@gmail.com> Signed-off-by: Lawrence Lane <llane@nvidia.com>

* ci: Bump version to 1.1.0 (#1364) (#1365) Signed-off-by: Dong Hyuk Chang <donghyukc@nvidia.com> Signed-off-by: NeMo Bot <nemo-bot@nvidia.com> Co-authored-by: Dong Hyuk Chang <thomaschang26@tutanota.com> Signed-off-by: Lawrence Lane <llane@nvidia.com> * feat: FFmpeg to 8.0.1 (#1362) (#1363) Signed-off-by: Ao Tang <aot@nvidia.com> Signed-off-by: NeMo Bot <nemo-bot@nvidia.com> Co-authored-by: Ao Tang <aot@nvidia.com> Co-authored-by: Ayush Dattagupta <ayushdg95@gmail.com> Signed-off-by: Lawrence Lane <llane@nvidia.com> * Fix bug in SDG example (#1370) (#1371) Signed-off-by: Sarah Yurick <sarahyurick@gmail.com> Signed-off-by: NeMo Bot <nemo-bot@nvidia.com> Co-authored-by: Sarah Yurick <53962159+sarahyurick@users.noreply.github.com> Signed-off-by: Lawrence Lane <llane@nvidia.com> * Fix bug in Gliner tutorial (#1372) (#1378) * Fix bug in Gliner tutorial * update readmes --------- Signed-off-by: Sarah Yurick <sarahyurick@gmail.com> Signed-off-by: NeMo Bot <nemo-bot@nvidia.com> Co-authored-by: Sarah Yurick <53962159+sarahyurick@users.noreply.github.com> Signed-off-by: Lawrence Lane <llane@nvidia.com> * Address aiohttp and urllib3 cve (#1379) (#1383) Signed-off-by: Dong Hyuk Chang <donghyukc@nvidia.com> Signed-off-by: NeMo Bot <nemo-bot@nvidia.com> Co-authored-by: Dong Hyuk Chang <thomaschang26@tutanota.com> Co-authored-by: Ayush Dattagupta <ayushdg95@gmail.com> Signed-off-by: Lawrence Lane <llane@nvidia.com> * Adding one worker per partition to FilePartioningStage and URLGeneratorStage (#1350) (#1366) Signed-off-by: Abhinav Garg <abhinavg@stanford.edu> Signed-off-by: NeMo Bot <nemo-bot@nvidia.com> Co-authored-by: Abhinav Garg <abhinavg@stanford.edu> Co-authored-by: Sarah Yurick <53962159+sarahyurick@users.noreply.github.com> Co-authored-by: Ayush Dattagupta <ayushdg95@gmail.com> Signed-off-by: Lawrence Lane <llane@nvidia.com> * Update instructions for AWS credentials in ArXiv download and extract tutorial (#1380) (#1402) * Update instructions for AWS credentials in ArXiv download and extract tutorial * ruff --------- Signed-off-by: Sarah Yurick <sarahyurick@gmail.com> Signed-off-by: NeMo Bot <nemo-bot@nvidia.com> Co-authored-by: Sarah Yurick <53962159+sarahyurick@users.noreply.github.com> Signed-off-by: Lawrence Lane <llane@nvidia.com> * cp: Revert "Remove nvenc/dec for xenna 0.1.6 (#1202)" (#1374) (#1403) This reverts commit c4805ae. Signed-off-by: Ayush Dattagupta <ayushdg95@gmail.com> Signed-off-by: Lawrence Lane <llane@nvidia.com> * Pin sklearn to < 1.8.0 for cuml 25.10 for r.1.1.0 #1405 Signed-off-by: Lawrence Lane <llane@nvidia.com> * Clarify instructions for downloading the Llama Nemotron Post-Training Dataset (#1416) (#1423) Signed-off-by: Sarah Yurick <sarahyurick@gmail.com> Signed-off-by: NeMo Bot <nemo-bot@nvidia.com> Co-authored-by: Sarah Yurick <53962159+sarahyurick@users.noreply.github.com> Signed-off-by: Lawrence Lane <llane@nvidia.com> * CP: Fix vllm API compatibility with Video Pipeline + Upgrade vLLM to 0.14 (#1429) * vllm API compatibility fixed Signed-off-by: Ao Tang <aot@nvidia.com> * upgrade vllm to 0.14.0 Signed-off-by: Ao Tang <aot@nvidia.com> * refactor Signed-off-by: Ao Tang <aot@nvidia.com> * pyproject update Signed-off-by: Ao Tang <aot@nvidia.com> * add protobuf in constraint-dependencies Signed-off-by: Ao Tang <aot@nvidia.com> * comment improve Signed-off-by: Ao Tang <aot@nvidia.com> * resolve pyproject Signed-off-by: Ao Tang <aot@nvidia.com> --------- Signed-off-by: Ao Tang <aot@nvidia.com> Signed-off-by: Lawrence Lane <llane@nvidia.com> * ci: Address setuptools CVE (#1438) (#1439) * Address CVE fixes * Remove cache of aiohttp from ray * Update uv lock * Update cache path --------- Signed-off-by: Dong Hyuk Chang <donghyukc@nvidia.com> Signed-off-by: Lawrence Lane <llane@nvidia.com> * ci: Optimize docker layer and uv with no cache (#1444) (#1446) * Optimize docker layer and uv with no cache * Add missing slash * Add comments to dockerfile --------- Signed-off-by: Dong Hyuk Chang <donghyukc@nvidia.com> Signed-off-by: NeMo Bot <nemo-bot@nvidia.com> Signed-off-by: Lawrence Lane <llane@nvidia.com> * Purge InternVideo2 (#1451) (#1462) * Remove Internvideo2 * more to remove * fix writer * Enhance Clip class to include cosmos_embed1_frames and cosmos_embed1_embedding in total size calculation * remove iv2 --------- Signed-off-by: Ao Tang <aot@nvidia.com> Signed-off-by: NeMo Bot <nemo-bot@nvidia.com> Co-authored-by: Ao Tang <aot@nvidia.com> Co-authored-by: Ayush Dattagupta <ayushdg95@gmail.com> Signed-off-by: Lawrence Lane <llane@nvidia.com> * ci: Update cve for python-multipart (#1450) (#1455) * Update cve for python-multipart * Update uv lock --------- Signed-off-by: Dong Hyuk Chang <donghyukc@nvidia.com> Co-authored-by: Sarah Yurick <53962159+sarahyurick@users.noreply.github.com> Signed-off-by: Lawrence Lane <llane@nvidia.com> * cherry pick commit, no benchmarking needed (#1461) Co-authored-by: Huy Vu2 <huvu@login-eos02.eos.clusters.nvidia.com> Signed-off-by: Lawrence Lane <llane@nvidia.com> * ci: Update vllm to 0.14.1 and override conflict (#1467) (#1468) * Update vllm to 0.14.1 and override conflict * Upperbound numpy for Numba compatibility * Update vllm to 0.15.1 --------- Signed-off-by: Dong Hyuk Chang <donghyukc@nvidia.com> Signed-off-by: NeMo Bot <nemo-bot@nvidia.com> Signed-off-by: Lawrence Lane <llane@nvidia.com> * ci: Remove thirdparty aiohttp file from ray (#1469) (#1475) Signed-off-by: Dong Hyuk Chang <donghyukc@nvidia.com> Signed-off-by: NeMo Bot <nemo-bot@nvidia.com> Signed-off-by: Lawrence Lane <llane@nvidia.com> * Fix: fasttext predict call for numpy>2 (#1482) (#1486) Signed-off-by: Ayush Dattagupta <ayushdg95@gmail.com> Signed-off-by: NeMo Bot <nemo-bot@nvidia.com> Co-authored-by: Ayush Dattagupta <ayushdg95@gmail.com> Signed-off-by: Lawrence Lane <llane@nvidia.com> * Update transformers dependency to exact version 4.55.2 in pyproject.t… (#1471) (#1488) * Update transformers dependency to exact version 4.55.2 in pyproject.toml and uv.lock to prevent import failures in Cosmos Embed. Downgrade tokenizers version to 0.21.4 for compatibility. * Update transformers dependency in pyproject.toml and uv.lock to allow versions up to 4.55.2, ensuring compatibility with Cosmos Embed imports. --------- Signed-off-by: Abhinav Garg <abhinavg@stanford.edu> Signed-off-by: NeMo Bot <nemo-bot@nvidia.com> Co-authored-by: Abhinav Garg <abhinavg@stanford.edu> Co-authored-by: Ayush Dattagupta <ayushdg95@gmail.com> Signed-off-by: Lawrence Lane <llane@nvidia.com> * Cherry pick `tutorials` changes from #1477 (#1491) * Update tutorial README Signed-off-by: Sarah Yurick <53962159+sarahyurick@users.noreply.github.com> * Update quickstart Updated sample sentences to provide more detailed feedback. Signed-off-by: Sarah Yurick <53962159+sarahyurick@users.noreply.github.com> --------- Signed-off-by: Sarah Yurick <53962159+sarahyurick@users.noreply.github.com> Signed-off-by: Lawrence Lane <llane@nvidia.com> * Update tutorial to be more explicit about num_gpus (#1492) (#1499) * Update tutorial to be more explicit about num_gpus * fix false positive secret scan * Hopefully fix the secrets --------- Signed-off-by: Ayush Dattagupta <ayushdg95@gmail.com> Signed-off-by: Dong Hyuk Chang <donghyukc@nvidia.com> Co-authored-by: Ayush Dattagupta <ayushdg95@gmail.com> Co-authored-by: Sarah Yurick <53962159+sarahyurick@users.noreply.github.com> Signed-off-by: Lawrence Lane <llane@nvidia.com> * Add relevant 26.02 docs to r1.1.0 (#1493) * add release notes Signed-off-by: Sarah Yurick <sarahyurick@gmail.com> * add more pages Signed-off-by: Sarah Yurick <sarahyurick@gmail.com> * add more pages Signed-off-by: Sarah Yurick <sarahyurick@gmail.com> * add new sdg docs Signed-off-by: Sarah Yurick <sarahyurick@gmail.com> * update remaining files from sdg docs Signed-off-by: Sarah Yurick <sarahyurick@gmail.com> * continue adding more changes Signed-off-by: Sarah Yurick <sarahyurick@gmail.com> * more video docs Signed-off-by: Sarah Yurick <sarahyurick@gmail.com> * add remaining updates Signed-off-by: Sarah Yurick <sarahyurick@gmail.com> --------- Signed-off-by: Sarah Yurick <sarahyurick@gmail.com> Signed-off-by: Lawrence Lane <llane@nvidia.com> * ci: Address new CVEs from rc4 (#1497) (#1500) * Scrub thirdparty aiohttp file from ray * Address new rc4 CVE * Apt get for consistency --------- Signed-off-by: Dong Hyuk Chang <donghyukc@nvidia.com> Signed-off-by: NeMo Bot <nemo-bot@nvidia.com> Co-authored-by: Ayush Dattagupta <ayushdg95@gmail.com> Signed-off-by: Lawrence Lane <llane@nvidia.com> * Add feedback to tutorials (#1476) (#1501) * Add feedback to tutorials * clarify install instructions for classifier tutorials * byo classifiers * add descriptions --------- Signed-off-by: Sarah Yurick <sarahyurick@gmail.com> Signed-off-by: Sarah Yurick <53962159+sarahyurick@users.noreply.github.com> Signed-off-by: NeMo Bot <nemo-bot@nvidia.com> Co-authored-by: Sarah Yurick <53962159+sarahyurick@users.noreply.github.com> Signed-off-by: Lawrence Lane <llane@nvidia.com> * ci: Update pyasn1 in uv lock (#1505) Signed-off-by: Dong Hyuk Chang <donghyukc@nvidia.com> Signed-off-by: Lawrence Lane <llane@nvidia.com> * Refactor video frame extraction to improve PyNvCodec availability check (#1511) (#1513) * Refactor video frame extraction to improve PyNvCodec availability check - Removed the try-except block for importing PyNvcFrameExtractor, simplifying the import logic. - Updated the condition for initializing the PyNvcFrameExtractor in the VideoFrameExtractionStage to rely solely on the _PYNVC_AVAILABLE flag. - Adjusted the handling of pixel format conversion in NvVideoDecoder to prepare for future updates to cvcuda. * Refactor NvVideoDecoder to replace deprecated nvcv_image with cvcuda tensor - Updated NvVideoDecoder to remove the use of nvcv_image, which is deprecated, and replaced it with cvcuda tensor. - Adjusted related tensor operations and tests to ensure compatibility with the new cvcuda implementation. * Update import statements in test_nvcodec_utils.py to include ruff linting rule - Modified import statements in the test file to include the RUF100 linting rule, ensuring better adherence to coding standards. - This change enhances the clarity of the import handling tests. * Update tests/utils/test_nvcodec_utils.py * Update tests/utils/test_nvcodec_utils.py --------- Signed-off-by: Abhinav Garg <abhinavg@stanford.edu> Signed-off-by: [Your Name] <your.email@example.com> Signed-off-by: NeMo Bot <nemo-bot@nvidia.com> Co-authored-by: Abhinav Garg <abhinavg@stanford.edu> Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> Co-authored-by: Ayush Dattagupta <ayushdg95@gmail.com> Signed-off-by: Lawrence Lane <llane@nvidia.com> * docs: isolate release notes and changelog (#1529) * docs: isolate release notes and changelog Signed-off-by: Lawrence Lane <llane@nvidia.com> * abhinav's feedback Signed-off-by: Lawrence Lane <llane@nvidia.com> * feedback Signed-off-by: Lawrence Lane <llane@nvidia.com> --------- Signed-off-by: Lawrence Lane <llane@nvidia.com> * ci: Update final release version (#1540) Signed-off-by: Dong Hyuk Chang <donghyukc@nvidia.com> Signed-off-by: Lawrence Lane <llane@nvidia.com> * docs: release note updates Signed-off-by: Lawrence Lane <llane@nvidia.com> --------- Signed-off-by: Dong Hyuk Chang <donghyukc@nvidia.com> Signed-off-by: NeMo Bot <nemo-bot@nvidia.com> Signed-off-by: Lawrence Lane <llane@nvidia.com> Signed-off-by: Ao Tang <aot@nvidia.com> Signed-off-by: Sarah Yurick <sarahyurick@gmail.com> Signed-off-by: Abhinav Garg <abhinavg@stanford.edu> Signed-off-by: Ayush Dattagupta <ayushdg95@gmail.com> Signed-off-by: Sarah Yurick <53962159+sarahyurick@users.noreply.github.com> Signed-off-by: [Your Name] <your.email@example.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Dong Hyuk Chang <thomaschang26@tutanota.com> Co-authored-by: Ao Tang <aot@nvidia.com> Co-authored-by: Ayush Dattagupta <ayushdg95@gmail.com> Co-authored-by: Sarah Yurick <53962159+sarahyurick@users.noreply.github.com> Co-authored-by: Abhinav Garg <abhinavg@stanford.edu> Co-authored-by: Praateek Mahajan <praateekmahajan@users.noreply.github.com> Co-authored-by: Huy Vu <86480512+huvunvidia@users.noreply.github.com> Co-authored-by: Huy Vu2 <huvu@login-eos02.eos.clusters.nvidia.com> Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

add release notes

2d014a3

Signed-off-by: Sarah Yurick <sarahyurick@gmail.com>

sarahyurick added 7 commits February 11, 2026 14:54

add more pages

9cdf9ef

Signed-off-by: Sarah Yurick <sarahyurick@gmail.com>

add more pages

2fdfe14

Signed-off-by: Sarah Yurick <sarahyurick@gmail.com>

add new sdg docs

4468ad3

Signed-off-by: Sarah Yurick <sarahyurick@gmail.com>

update remaining files from sdg docs

238a8d4

Signed-off-by: Sarah Yurick <sarahyurick@gmail.com>

continue adding more changes

bda3993

Signed-off-by: Sarah Yurick <sarahyurick@gmail.com>

more video docs

18bfe0b

Signed-off-by: Sarah Yurick <sarahyurick@gmail.com>

add remaining updates

00ec828

Signed-off-by: Sarah Yurick <sarahyurick@gmail.com>

sarahyurick marked this pull request as ready for review February 12, 2026 00:08

sarahyurick requested review from lbliii and thomasdhc February 12, 2026 00:10

greptile-apps Bot reviewed Feb 12, 2026

View reviewed changes

lbliii approved these changes Feb 12, 2026

View reviewed changes

Merge branch 'r1.1.0' into sarahyurick/docs

64ccf9a

sarahyurick merged commit f1e0a2c into r1.1.0 Feb 12, 2026
18 checks passed

greptile-apps Bot reviewed Feb 12, 2026

View reviewed changes

Navigation Menu

Uh oh!

Add relevant 26.02 docs to r1.1.0 - #1493

Add relevant 26.02 docs to r1.1.0#1493
sarahyurick merged 9 commits into
r1.1.0from
sarahyurick/docs

sarahyurick commented Feb 11, 2026 •

edited

Loading

Uh oh!

copy-pr-bot Bot commented Feb 11, 2026

Uh oh!

greptile-apps Bot commented Feb 12, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot left a comment

Uh oh!

Uh oh!

greptile-apps Bot left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

sarahyurick commented Feb 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

copy-pr-bot Bot commented Feb 11, 2026

Uh oh!

greptile-apps Bot commented Feb 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Overview

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Uh oh!

greptile-apps Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

greptile-apps Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

sarahyurick commented Feb 11, 2026 •

edited

Loading

greptile-apps Bot commented Feb 12, 2026 •

edited

Loading