NVIDIA-NeMo · lbliii · Apr 3, 2026
@@ -459,6 +459,15 @@ navigation:
                   - page: Format Validation
                     path: ./v26.04/pages/curate-audio/process-data/audio-analysis/format-validation.mdx
                     slug: format-validation
+              - section: Preprocessing
+                slug: preprocessing
+                contents:
+                  - page: Overview
+                    path: ./v26.04/pages/curate-audio/process-data/preprocessing/index.mdx
+                    slug: ""
+                  - page: Band Filtering
+                    path: ./v26.04/pages/curate-audio/process-data/preprocessing/band-filtering.mdx
+                    slug: band-filtering
               - page: Text Integration
                 path: ./v26.04/pages/curate-audio/process-data/text-integration/index.mdx
                 slug: text-integration

@@ -62,6 +62,54 @@ Standardized return type for all deduplication workflows:
 - **Structured metadata**: Access per-stage timing (`total_time`, `identification_time`, and others), duplicate counts (`num_duplicates`, `num_duplicates_removed`), and output paths through `result.metadata`
 - **`TaskPerfUtils` compatibility**: `collect_stage_metrics()` and `aggregate_task_metrics()` now accept `WorkflowRunResult` directly
 
+### Nemotron VLM for Video Captioning (PR #1160)
+
+Added NemotronH hybrid Mamba-Attention VLM as an alternative to Qwen-VL for video captioning:
+
+- **`NemotronHVL` model class**: Supports `nemotron` (BF16), `nemotron-fp8`, and `nemotron-nvfp4` quantization variants from HuggingFace (`nvidia/NVIDIA-Nemotron-Nano-12B-v2-VL-*`).
+- **`CaptionGenerationStage` and `CaptionPreparationStage`**: Updated to accept `model_variant="nemotron"` alongside the existing `"qwen"` option.
+- **Automatic model download**: Models download from HuggingFace on first use with pinned revision hashes for reproducibility.
+
+Learn more in the [Captions and Preview](/curate-video/process-data/captions-preview) documentation.
+
+### Audio Preprocessing Stages (PR #1575)
+
+New stages for preparing raw audio before ASR and quality assessment:
+
+- **`MonoConversionStage`**: Converts multi-channel audio to mono by averaging channels and verifies sample rate compliance.
+- **`SegmentConcatenationStage`**: Assembles VAD-segmented audio (from `VADSegmentationStage(nested=True)`) into a single waveform per source file with configurable silence between segments.
+- **`TimestampMapperStage`**: Resolves segment positions in the concatenated waveform back to original file timestamps, producing metadata-only output.
+
+Learn more in the [Audio Preprocessing](/curate-audio/process-data/preprocessing) documentation.
+
+### Audio Band Filtering (PR #1576)
+
+New bandwidth classification stage for filtering audio by spectral quality:
+
+- **`BandFilterStage`**: Classifies audio as `"full_band"` or `"narrow_band"` using a pre-trained model from HuggingFace (`nvidia/nemocurator-speech-bandwidth-filter`), then filters based on the specified band type.
+- **GPU support**: Configurable via `Resources(gpus=N)` for GPU-accelerated inference.
+
+Learn more in the [Band Filtering](/curate-audio/process-data/preprocessing/band-filtering) documentation.
+
+### Fuzzy Dedup LSH Memory Tuning (PR #1603)
+
+Added GPU memory management parameters to `FuzzyDeduplicationWorkflow` for the LSH stage:
+
+- **`lsh_num_output_partitions`**: Control the number of shuffle output partitions (default: auto-selected as closest power of 2 &lt;= input tasks).
+- **`lsh_rmm_pool_size`**: Configure the RMM GPU memory pool size (`"auto"` = 90% of free memory, `None` = 50% expandable).
+- **`lsh_spill_memory_limit`**: Set the device memory spill threshold (`"auto"` = 80% of RMM pool, `None` = disabled).
+
+Learn more in the [Fuzzy Deduplication](/curate-text/process-data/deduplication/fuzzy) documentation.
+
+### Interleaved IO Improvements (PR #1652)
+
+Schema utilities and base-class enhancements for multimodal readers and writers:
+
+- **`schema` and `schema_overrides` parameters**: New parameters on `BaseInterleavedReader` for strict schema alignment or partial type overrides on output tables.
+- **`WebdatasetReaderStage` renamed to `InterleavedWebdatasetReaderStage`**: Clarifies naming for the interleaved reader stage.
+- **`source_id_field` removed**: The `source_id_field` parameter has been removed from `WebdatasetReader` and `InterleavedWebdatasetReaderStage`.
+- **Schema utility functions**: `reconcile_schema`, `resolve_schema`, and `align_table` handle type reconciliation for reserved and passthrough columns.
+
 ### Actor Pool Progress Bars (PR #1457)
 
 Added tqdm progress bars to `RayActorPoolExecutor` for real-time visibility into task completion during stage processing and shuffle inserts. Progress bars are enabled by default and can be configured with `show_progress` and `progress_interval` parameters. This is particularly useful for long-running deduplication jobs where progress is not otherwise apparent.
@@ -72,6 +120,11 @@ Added tqdm progress bars to `RayActorPoolExecutor` for real-time visibility into
 - **Ray**: Updated to 2.54
 - **cryptography**: Bumped from >=46.0.5 to >=46.0.6 to address CVE GHSA-m959-cc7f-wv43 (PR #1682)
 - **uv**: Added minimum required version (>=0.7.0) to prevent lockfile revision drift
+- **nemo-toolkit[asr]**: Bumped to >=2.7.2 to fix CVE-2025-33245 and CVE-2025-33253 (RCE via unsafe deserialization) (PR #1612)
+- **xgrammar**: Bumped to >=0.1.32 to fix GHSA-7rgv-gqhr-fxg3 (DoS via uncontrolled recursion) (PR #1612)
+- **pyopenssl**: Bumped from 22.0.0 to 26.0.0 (PR #1669)
+- **onnx**: Bumped from 1.20.0 to 1.21.0 (PR #1713)
+- **flash-attn**: Pinned to >=2.8.3 for NemotronH VLM compatibility (PR #1160)
 
 ## Bug Fixes
 
@@ -95,3 +148,7 @@ Fixed a race condition in `CaptionGenerationStage` and `CaptionEnhancementStage`
 - **`SemanticDeduplicationWorkflow.run()` and `TextSemanticDeduplicationWorkflow.run()`** now return `WorkflowRunResult` instead of `dict`
 - **`TextDuplicatesRemovalWorkflow.run()`** now returns `WorkflowRunResult` instead of `list[FileGroupTask] | None`
 - Code that previously ignored the return value is unaffected. Code that consumed the old `dict` return from semantic workflows must migrate to `result.metadata` and `result.pipeline_tasks`.
+- **`WebdatasetReaderStage` renamed to `InterleavedWebdatasetReaderStage`** (PR #1652): Code that imports `WebdatasetReaderStage` directly must update to `InterleavedWebdatasetReaderStage`. The `WebdatasetReader` composite stage is unchanged.
+- **`source_id_field` removed** (PR #1652): The `source_id_field` parameter has been removed from `WebdatasetReader` and the underlying reader stage. Remove this parameter from existing code.
+- **`ray_dist.jar` removed from Docker image** (PR #1612): The `ray_dist.jar` file has been removed from the Dockerfile to address GHSA-72hv-8253-57qq. This does not affect Python-based workflows.
+- **`FuzzyDeduplicationWorkflow` LSH defaults changed** (PR #1603): The `rmm_pool_size` and `spill_memory_limit` parameters passed to the internal LSH stage now default to `"auto"` (previously hardcoded). This is exposed through the new `lsh_rmm_pool_size` and `lsh_spill_memory_limit` workflow parameters.
@@ -0,0 +1,97 @@
+---
+description: "Classify and filter audio by bandwidth using a pre-trained model to remove low-quality narrow-band recordings"
+categories: ["audio-processing"]
+tags: ["audio-filtering", "bandwidth-classification", "band-filter", "quality-filtering"]
+personas: ["data-scientist-focused", "mle-focused"]
+difficulty: "beginner"
+content_type: "how-to"
+modality: "audio-only"
+---
+
+# Band Filtering
+
+Classify audio as full-band or narrow-band based on spectral characteristics, then filter to keep only the desired bandwidth type. This is useful for removing low-quality telephone or compressed audio from training datasets.
+
+## How It Works
+
+`BandFilterStage` uses a pre-trained classifier model to predict whether each audio file is full-band or narrow-band:
+
+- **Full-band**: Broad frequency range typical of high-quality recordings
+- **Narrow-band**: Limited frequency range typical of telephone audio or heavy compression
+
+The model downloads automatically from HuggingFace (`nvidia/nemocurator-speech-bandwidth-filter`) on first use.
+
+## Quick Start
+
+```python
+from nemo_curator.pipeline import Pipeline
+from nemo_curator.stages.audio.filtering import BandFilterStage
+
+pipeline = Pipeline(name="band_filter")
+
+# Keep only full-band audio
+pipeline.add_stage(BandFilterStage(band_value="full_band"))
+```
+
+## Configuration
+
+| Parameter | Type | Default | Description |
+| --- | --- | --- | --- |
+| `model_path` | str \| None | None | Local path to `.joblib` model file. Auto-downloads from HuggingFace if not provided. |
+| `cache_dir` | str \| None | None | Directory to cache downloaded models |
+| `band_value` | "full_band" \| "narrow_band" | "full_band" | Which bandwidth type to pass through the filter |
+
+**Output field**: `band_prediction` (str: "full_band" or "narrow_band")
+
+## GPU Support
+
+`BandFilterStage` runs on CPU by default. To use GPU acceleration, configure the stage resources:
+
+```python
+from nemo_curator.stages.audio.filtering import BandFilterStage
+from nemo_curator.stages.resources import Resources
+
+stage = BandFilterStage(
+    band_value="full_band",
+    resources=Resources(gpus=1),
+)
+```
+
+## Using a Local Model
+
+If you have a pre-downloaded model or a custom-trained classifier, provide the path directly:
+
+```python
+stage = BandFilterStage(
+    model_path="/path/to/band_classifier_model.joblib",
+    band_value="full_band",
+)
+```
+
+## Pipeline Example
+
+Combine band filtering with other audio processing stages:
+
+```python
+from nemo_curator.pipeline import Pipeline
+from nemo_curator.stages.audio.filtering import BandFilterStage
+from nemo_curator.stages.audio.common import GetAudioDurationStage, PreserveByValueStage
+
+pipeline = Pipeline(name="audio_quality")
+
+# 1. Calculate duration
+pipeline.add_stage(GetAudioDurationStage(
+    audio_filepath_key="audio_filepath",
+    duration_key="duration",
+))
+
+# 2. Filter short/long audio
+pipeline.add_stage(PreserveByValueStage(
+    input_value_key="duration",
+    target_value=1.0,
+    operator="ge",
+))
+
+# 3. Keep only full-band audio
+pipeline.add_stage(BandFilterStage(band_value="full_band"))
+```
@@ -0,0 +1,127 @@
+---
+description: "Convert multi-channel audio to mono, concatenate VAD segments, and map timestamps back to original files"
+categories: ["audio-processing"]
+tags: ["audio-preprocessing", "mono-conversion", "segment-concatenation", "timestamp-mapping", "vad"]
+personas: ["data-scientist-focused", "mle-focused"]
+difficulty: "intermediate"
+content_type: "how-to"
+modality: "audio-only"
+---
+
+# Audio Preprocessing
+
+Prepare raw audio files for downstream processing with stages that handle channel conversion, segment assembly, and timestamp resolution. These stages typically run early in an audio pipeline before ASR inference and quality assessment.
+
+## Overview
+
+Audio preprocessing in NeMo Curator provides three stages that work together:
+
+1. **MonoConversionStage**: Converts multi-channel audio to mono and verifies sample rate
+2. **SegmentConcatenationStage**: Assembles VAD-segmented audio into a single waveform per source file
+3. **TimestampMapperStage**: Resolves segment positions in the concatenated waveform back to the original file
+
+A typical preprocessing pipeline chains these stages with VAD segmentation:
+
+```
+MonoConversion → VADSegmentation(nested=True) → SegmentConcatenation → [processing] → TimestampMapper
+```
+
+## Mono Conversion
+
+Convert multi-channel audio to mono by averaging channels and verify the sample rate matches expectations.
+
+```python
+from nemo_curator.stages.audio.preprocessing import MonoConversionStage
+
+stage = MonoConversionStage(
+    output_sample_rate=48000,
+    audio_filepath_key="audio_filepath",
+    strict_sample_rate=True,
+)
+```
+
+| Parameter | Type | Default | Description |
+| --- | --- | --- | --- |
+| `output_sample_rate` | int | 48000 | Expected sample rate in Hz |
+| `audio_filepath_key` | str | "audio_filepath" | Key in data dict for audio file path |
+| `strict_sample_rate` | bool | True | Reject audio with mismatched sample rate |
+
+**Output fields**: `waveform`, `sample_rate`, `is_mono`, `duration`, `num_samples`
+
+When `strict_sample_rate` is `True`, audio files with a sample rate different from `output_sample_rate` are filtered out. Set to `False` to accept any sample rate.
+
+## Segment Concatenation
+
+Concatenate VAD segments from `VADSegmentationStage(nested=True)` into a single waveform per source file, with configurable silence between segments.
+
+```python
+from nemo_curator.stages.audio.preprocessing import SegmentConcatenationStage
+
+stage = SegmentConcatenationStage(
+    silence_duration_sec=0.5,
+)
+```
+
+| Parameter | Type | Default | Description |
+| --- | --- | --- | --- |
+| `silence_duration_sec` | float | 0.5 | Duration of silence inserted between segments (seconds) |
+
+**Input**: Expects `task.data["segments"]` list from `VADSegmentationStage(nested=True)`
+
+**Output fields**: `waveform`, `sample_rate`, `num_segments`, `total_duration_sec`, `original_file`
+
+Segments are sorted by `segment_num` before concatenation. Gaps from filtered-out segments are preserved in the correct order. Segment-to-original mappings are stored in `task._metadata["segment_mappings"]` for use by `TimestampMapperStage`.
+
+## Timestamp Mapping
+
+Resolve segment positions in the concatenated waveform back to positions in the original audio file. Strips waveform data from the output, producing metadata-only results.
+
+```python
+from nemo_curator.stages.audio.postprocessing import TimestampMapperStage
+
+stage = TimestampMapperStage(
+    passthrough_keys=["speaker_id", "quality_score"],
+)
+```
+
+| Parameter | Type | Default | Description |
+| --- | --- | --- | --- |
+| `passthrough_keys` | list[str] \| None | None | Additional data keys to preserve in output |
+
+**Input**: Requires `task._metadata["segment_mappings"]` from `SegmentConcatenationStage`
+
+**Output fields**: `original_file`, `original_start_ms`, `original_end_ms`, `duration_ms`, `duration_sec`
+
+## Complete Pipeline
+
+Here is a complete pipeline combining preprocessing stages with VAD and ASR:
+
+```python
+from nemo_curator.pipeline import Pipeline
+from nemo_curator.stages.audio.preprocessing import MonoConversionStage, SegmentConcatenationStage
+from nemo_curator.stages.audio.postprocessing import TimestampMapperStage
+
+pipeline = Pipeline(name="audio_preprocessing")
+
+# 1. Convert to mono and verify sample rate
+pipeline.add_stage(MonoConversionStage(
+    output_sample_rate=48000,
+    strict_sample_rate=True,
+))
+
+# 2. Run VAD segmentation (nested mode groups segments per file)
+# pipeline.add_stage(VADSegmentationStage(nested=True))
+
+# 3. Concatenate segments into single waveform
+pipeline.add_stage(SegmentConcatenationStage(
+    silence_duration_sec=0.5,
+))
+
+# 4. Run downstream processing (ASR, quality assessment, etc.)
+# ...
+
+# 5. Map timestamps back to original file positions
+pipeline.add_stage(TimestampMapperStage(
+    passthrough_keys=["transcript", "quality_score"],
+))
+```
@@ -95,6 +95,29 @@ Configure fuzzy deduplication using these key parameters:
 | `input_filetype` | str | "parquet" | Input file format ("parquet" or "jsonl") |
 | `input_blocksize` | str \| int | "1GiB" | Size of input blocks for processing |
 | `perform_removal` | bool | False | Reserved; must remain `False`. Fuzzy removal is performed with `TextDuplicatesRemovalWorkflow`. |
+| `lsh_num_output_partitions` | int \| None | None | Total partitions during LSH shuffle. If None, auto-selects closest power of 2 &lt;= input tasks. |
+| `lsh_rmm_pool_size` | int \| "auto" \| None | "auto" | RMM GPU memory pool size in bytes. `"auto"` = 90% of free GPU memory; `None` = 50% expandable. |
+| `lsh_spill_memory_limit` | int \| "auto" \| None | "auto" | Device memory spill limit in bytes. `"auto"` = 80% of RMM pool; `None` = spilling disabled. |
+
+### LSH GPU Memory Tuning
+
+The `lsh_*` parameters control GPU memory allocation during the LSH stage, which is the most memory-intensive phase:
+
+- **`lsh_rmm_pool_size`**: Controls the RAPIDS Memory Manager pool. `"auto"` (default) allocates 90% of free GPU memory as a fixed pool. Set to `None` for a 50% expandable pool that grows on demand.
+- **`lsh_spill_memory_limit`**: Controls when data spills from GPU to host memory. `"auto"` (default) spills at 80% of the RMM pool. Set to `None` to disable spilling entirely (risks OOM on large datasets).
+- **`lsh_num_output_partitions`**: Controls shuffle parallelism. Higher values reduce per-partition memory at the cost of more shuffle overhead. Leave as `None` for automatic selection.
+
+```python
+# Example: manual GPU memory configuration for a 40GB GPU
+fuzzy_workflow = FuzzyDeduplicationWorkflow(
+    input_path="input_data/",
+    cache_path="./cache",
+    output_path="./results",
+    lsh_rmm_pool_size=34_000_000_000,     # 34GB fixed pool
+    lsh_spill_memory_limit=27_000_000_000, # Spill at 27GB
+    lsh_num_output_partitions=64,          # 64 output partitions
+)
+```
 
 ### Similarity Threshold