Skip to content

docs: add fern docs for 26.04 PRs (#1160, #1575, #1576, #1603, #1612, #1652) - #1730

Closed
lbliii wants to merge 1 commit into
NVIDIA-NeMo:26.04-stagingfrom
lbliii:lbliii/fern-docs-2604
Closed

docs: add fern docs for 26.04 PRs (#1160, #1575, #1576, #1603, #1612, #1652)#1730
lbliii wants to merge 1 commit into
NVIDIA-NeMo:26.04-stagingfrom
lbliii:lbliii/fern-docs-2604

Conversation

@lbliii

@lbliii lbliii commented Apr 3, 2026

Copy link
Copy Markdown
Contributor

Description

Add fern documentation and release notes for features merged to main but not yet documented on 26.04-staging. Covers audio preprocessing stages (MonoConversion, SegmentConcatenation, TimestampMapper from PR #1575), audio band filtering (BandFilterStage from PR #1576), Nemotron VLM video captioning support (PR #1160), fuzzy dedup LSH memory tuning params (PR #1603), interleaved IO schema improvements (PR #1652), and dependency/security updates (PR #1612). Includes new pages, updated existing pages, nav updates, and release note line items with breaking changes.

Checklist

  • I am familiar with the Contributing Guide.
  • New or Existing tests cover these changes.
  • The documentation is up to date with these changes.

…NVIDIA-NeMo#1160, NVIDIA-NeMo#1603, NVIDIA-NeMo#1612, NVIDIA-NeMo#1652)

Add audio preprocessing docs (MonoConversion, SegmentConcatenation,
TimestampMapper), band filtering docs, Nemotron VLM video captioning
updates, fuzzy dedup LSH memory tuning params, interleaved IO schema
changes, and release note entries for all new features, dependency
updates, and breaking changes.

Signed-off-by: Lawrence Lane <llane@nvidia.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Lawrence Lane <llane@nvidia.com>
@lbliii
lbliii requested a review from a team as a code owner April 3, 2026 15:23
@lbliii
lbliii requested review from weijiac0619 and removed request for a team April 3, 2026 15:23
@greptile-apps

greptile-apps Bot commented Apr 3, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This documentation-only PR backfills Fern docs and release notes for six features that were merged to main but not yet documented on the 26.04-staging branch. It adds two new MDX pages (audio preprocessing overview and band filtering), updates the video captioning page to cover Nemotron VLM alongside Qwen-VL, adds LSH GPU memory-tuning parameters to the fuzzy deduplication page, updates the navigation config, and adds corresponding release-note entries including security CVE references and breaking changes.

Key changes:

  • New fern/versions/v26.04/pages/curate-audio/process-data/preprocessing/ section covering MonoConversionStage, SegmentConcatenationStage, and TimestampMapperStage
  • New band-filtering.mdx documenting BandFilterStage with CPU/GPU and local-model usage
  • captions-preview.mdx updated with a Nemotron VLM tab, variant table, and updated parameter descriptions — two prose sentences in the "Preparation" and "Caption generation" sections still reference Qwen-specific output keys and model names and were not updated alongside the new content
  • fuzzy.mdx extended with lsh_num_output_partitions, lsh_rmm_pool_size, and lsh_spill_memory_limit parameter docs
  • Release notes correctly cover all six PRs, including breaking changes (WebdatasetReaderStage rename, source_id_field removal, LSH default changes) and security dependency bumps

Confidence Score: 4/5

Safe to merge; documentation-only changes with no runtime risk, minor stale prose in captions-preview.mdx.

All changes are MDX documentation files and a YAML navigation config — no executable code is modified. The two stale Qwen-specific references in captions-preview.mdx are minor inaccuracies that won't break anything but could confuse Nemotron VLM users. Everything else is accurate and well-structured.

fern/versions/v26.04/pages/curate-video/process-data/captions-preview.mdx — two prose sections still mention Qwen-specific output keys after Nemotron VLM support was added.

Important Files Changed

Filename Overview
fern/versions/v26.04/pages/curate-video/process-data/captions-preview.mdx Nemotron VLM support added, but two prose sections in "Preparation and previews" and "Caption generation and enhancement" still reference Qwen-specific output keys and model name, creating stale documentation.
fern/versions/v26.04/pages/curate-audio/process-data/preprocessing/index.mdx New audio preprocessing overview page; TimestampMapperStage is imported from nemo_curator.stages.audio.postprocessing while MonoConversionStage/SegmentConcatenationStage come from preprocessing — worth verifying against actual module locations.
fern/versions/v26.04/pages/about/release-notes/index.mdx Well-structured release notes covering six PRs; security CVE references, breaking changes (rename, removed parameter, default changes), and new features are clearly documented.
fern/versions/v26.04/pages/curate-text/process-data/deduplication/fuzzy.mdx Clean addition of three new LSH memory-tuning parameters with good explanation and a concrete 40GB GPU example; existing content untouched.
fern/versions/v26.04/pages/curate-audio/process-data/preprocessing/band-filtering.mdx New band filtering documentation; comprehensive coverage of configuration, GPU usage, local model usage, and pipeline example with clear parameter tables.
fern/versions/v26.04.yml Navigation config correctly adds the new Preprocessing section with Overview and Band Filtering pages under curate-audio.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Raw Audio Files] --> B[MonoConversionStage\nConvert to mono, verify sample rate]
    B -->|strict_sample_rate=True| C{Sample rate\nmatch?}
    C -->|No| D[Filtered out]
    C -->|Yes| E[VADSegmentationStage\nnested=True]
    E --> F[SegmentConcatenationStage\nAssemble waveform, insert silence between segments]
    F --> G[Downstream Processing\nASR, Quality Assessment, BandFilterStage, etc.]
    G --> H[TimestampMapperStage\nResolve segment positions to original timestamps]
    H --> I[Metadata Output\noriginal_file, original_start_ms, original_end_ms]

    style D fill:#f88,stroke:#c00
    style I fill:#8f8,stroke:#090
Loading

Comments Outside Diff (2)

  1. fern/versions/v26.04/pages/curate-video/process-data/captions-preview.mdx, line 122 (link)

    P2 Stale Qwen-only reference in preparation step description

    This prose description still says "formats model‑ready inputs for Qwen‑VL" even though CaptionPreparationStage now supports both "qwen" and "nemotron" variants. A reader following the Nemotron VLM tab above would find this description contradictory.

  2. fern/versions/v26.04/pages/curate-video/process-data/captions-preview.mdx, line 208 (link)

    P2 Hardcoded Qwen output-key references are now stale

    This sentence hard-codes Qwen-specific internal output keys (window.caption["qwen"] and window.enhanced_caption["qwen_lm"]). Since CaptionEnhancementStage now also accepts model_variant="nemotron" (as shown in the parameters table added in this PR), a reader using the Nemotron path will find these key names confusing or incorrect. Consider generalising the description to be model-agnostic, similar to how step 1 was updated:

Reviews (1): Last reviewed commit: "docs: add fern docs for 26.04 PRs (#1575..." | Re-trigger Greptile

@lbliii lbliii closed this Apr 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant