Skip to content

docs: rename AudioBatch to AudioTask in audio curation docs #1694

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
lbliii merged 5 commits into from
Apr 14, 2026
Merged

docs: rename AudioBatch to AudioTask in audio curation docs #1694

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
86bebb2
docs: rename AudioBatch to AudioTask across audio curation docs
lbliii Mar 31, 2026
61bb185
docs: fix style guide issues in audio curation docs
lbliii Mar 31, 2026
a7ea2af
merge: resolve conflicts with 26.04-staging
lbliii Apr 6, 2026
9a2f327
docs: fix AudioTask API mismatches flagged in PR review
lbliii Apr 14, 2026
6164494
merge: resolve conflicts with 26.04-staging
lbliii Apr 14, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 17 additions & 1 deletion fern/docs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,10 +51,26 @@ redirects:
- source: "/nemo/curator/:path*.html"
destination: "/nemo/curator/:path*"
# Audio concepts: legacy nested nav URLs → flat slugs (aligned with Sphinx filenames)
- source: "/about/concepts/audio/curation/pipeline"
destination: "/about/concepts/audio/curation-pipeline"
- source: "/about/concepts/audio/audio/batch"
destination: "/about/concepts/audio/audio-task"
- source: "/about/concepts/audio/audio-batch"
destination: "/about/concepts/audio/audio-task"
- source: "/about/concepts/audio/asr/pipeline"
destination: "/about/concepts/audio/asr-pipeline"
- source: "/about/concepts/audio/quality/metrics"
destination: "/about/concepts/audio/quality-metrics"
- source: "/about/concepts/audio/manifests/ingest"
destination: "/about/concepts/audio/manifests-ingest"
- source: "/about/concepts/audio/text/integration"
destination: "/about/concepts/audio/text-integration"
- source: "/nemo/curator/about/concepts/audio/curation/pipeline"
destination: "/nemo/curator/about/concepts/audio/curation-pipeline"
- source: "/nemo/curator/about/concepts/audio/audio/batch"
destination: "/nemo/curator/about/concepts/audio/audio-batch"
destination: "/nemo/curator/about/concepts/audio/audio-task"
- source: "/nemo/curator/about/concepts/audio/audio-batch"
destination: "/nemo/curator/about/concepts/audio/audio-task"
- source: "/nemo/curator/about/concepts/audio/asr/pipeline"
destination: "/nemo/curator/about/concepts/audio/asr-pipeline"
- source: "/nemo/curator/about/concepts/audio/quality/metrics"
Expand Down
12 changes: 6 additions & 6 deletions fern/versions/v26.04.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,9 +96,9 @@ navigation:
- page: Curation Pipeline
path: ./v26.04/pages/about/concepts/audio/curation-pipeline.mdx
slug: curation-pipeline
- page: Audio Batch
path: ./v26.04/pages/about/concepts/audio/audio-batch.mdx
slug: audio-batch
- page: Audio Task
path: ./v26.04/pages/about/concepts/audio/audio-task.mdx
slug: audio-task
- page: ASR Pipeline
path: ./v26.04/pages/about/concepts/audio/asr-pipeline.mdx
slug: asr-pipeline
Expand Down Expand Up @@ -555,9 +555,9 @@ navigation:
- page: VideoTask
path: ./v26.04/pages/api-reference/tasks/video-task.mdx
slug: video-task
- page: AudioBatch
path: ./v26.04/pages/api-reference/tasks/audio-batch.mdx
slug: audio-batch
- page: AudioTask
path: ./v26.04/pages/api-reference/tasks/audio-task.mdx
slug: audio-task
- section: Executors
slug: executors
contents:
Expand Down
22 changes: 11 additions & 11 deletions fern/versions/v26.04/pages/about/concepts/audio/asr-pipeline.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ The ASR pipeline in NeMo Curator follows a systematic approach to speech process

```mermaid
graph TD
A[Audio Files] --> B[AudioBatch Creation]
A[Audio Files] --> B[AudioTask Creation]
B --> C[ASR Model Loading]
C --> D[Batch Inference]
D --> E[Transcription Output]
Expand Down Expand Up @@ -46,15 +46,15 @@ graph TD

### 1. Audio Input Management

**AudioBatch Structure**: The foundation for audio processing
**AudioTask Structure**: The foundation for audio processing

- Contains audio file paths and associated metadata
- Validates file existence and accessibility automatically
- Supports efficient batch processing for scalability

**Input Validation**: Ensures data integrity before processing

- File path existence checks using `AudioBatch.validate()` and `validate_item()`
- File path existence checks using `AudioTask.validate()` and `validate_item()`
- Optional metadata validation added by downstream stages (such as duration and format checks)

### 2. ASR Model Integration
Expand Down Expand Up @@ -104,7 +104,7 @@ manifest_reader = JsonlReader(
input_file_path="/path/to/manifest.jsonl"
)

# Stages automatically create AudioBatch objects from loaded data
# Stages automatically create AudioTask objects from loaded data
```

### Stage 2: ASR Model Setup
Expand All @@ -125,11 +125,11 @@ asr_stage.setup() # Downloads and loads model
### Stage 3: Transcription Generation

```python
# ASR stage processes AudioBatch objects automatically
# ASR stage processes AudioTask objects automatically
# The stage extracts file paths and calls transcribe() internally
processed_batch = asr_stage.process(audio_batch)
processed_batch = asr_stage.process(audio_task)

@mohammadaaftabv mohammadaaftabv Apr 8, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

InferenceAsrNemoStage.process() raises NotImplementedError.
Users should never call process() directly on ASR stages — the pipeline executor handles dispatch via process_batch().

Fix — either show process_batch() directly:

results = asr_stage.process_batch([audio_task])
Or (better) show the executor-driven pattern:

Don't call process() directly — the Pipeline/Executor handles dispatch:

pipeline.add_stage(asr_stage)
results = pipeline.run(executor)


# Output: AudioBatch with added "pred_text" field
# Output: AudioTask with added "pred_text" field
# Each item now contains both original data and predictions
```

Expand All @@ -156,7 +156,7 @@ duration_stage = GetAudioDurationStage(

1. **Audio Files** → File system
2. **Manifest Files** → JSONL format with metadata
3. **AudioBatch Objects** → Validated, structured data containers
3. **AudioTask Objects** → Validated, structured data containers

### Processing Data Flow

Expand Down Expand Up @@ -217,10 +217,10 @@ asr_stage = InferenceAsrNemoStage(

```python
# Validate and filter invalid file paths
audio_batch = AudioBatch(data=audio_data, filepath_key="audio_filepath")
audio_task = AudioTask(data=audio_data, filepath_key="audio_filepath")

# Filter out entries that do not exist on disk
valid_samples = [item for item in audio_batch.data if audio_batch.validate_item(item)]
valid_samples = [item for item in audio_task.data if audio_task.validate_item(item)]

@greptile-apps greptile-apps Bot Mar 31, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Iterating over AudioTask.data yields dict keys, not items

AudioTask.data is a single dict (or _AttrDict), so for item in audio_task.data iterates over the dict's keys (strings like "audio_filepath", "text", etc.) rather than over audio sample entries. The original AudioBatch.data was a list[dict], which is why the list comprehension made sense before.

Since AudioTask holds exactly one manifest entry, the validation should be called on that single entry directly — for example, audio_task.validate_item(audio_task.data). If the intent is to show bulk filtering across many tasks, each task should be constructed individually with one dict, then filtered in a loop.

@mohammadaaftabv mohammadaaftabv Apr 8, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@mohammadaaftabv mohammadaaftabv Apr 8, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

audio_task.data is a dict (specifically _AttrDict). Iterating over a dict yields its keys — strings like "audio_filepath", "text", "duration" — not manifest entries. This is holdover from AudioBatch.data which was list[dict].

The comprehension would produce something like ["audio_filepath", "text"] (keys that happen to pass validate_item), which is nonsensical.

Since AudioTask is single-entry, validation is simply:

is_valid = audio_task.validate()

```

### Pipeline Recovery
Expand All @@ -237,7 +237,7 @@ The ASR pipeline seamlessly integrates with text processing workflows:
# Audio → Text pipeline
audio_to_text = [
InferenceAsrNemoStage(), # Audio → Transcriptions
AudioToDocumentStage(), # AudioBatch → DocumentBatch
AudioToDocumentStage(), # AudioTask → DocumentBatch
# Continue with text processing stages...
]
```
249 changes: 0 additions & 249 deletions fern/versions/v26.04/pages/about/concepts/audio/audio-batch.mdx
View file
Open in desktop

This file was deleted.

Loading
Loading