Add ADV audio processing dependencies by shubhamNvidia · Pull Request #1574 · NVIDIA-NeMo/Curator

shubhamNvidia · 2026-03-05T14:41:04Z

Adds the audio_common dependency group with packages required by the Audio DataVerse (ADV) pipeline stages: silero-vad, onnx, onnxruntime/onnxruntime-gpu, nvidia-cudnn-cu12, torchaudio, Cython, seaborn, torchcodec, and packaging. Splits audio_cpu and audio_cuda12 to properly separate CPU-only and GPU-accelerated ONNX runtime variants.

copy-pr-bot · 2026-03-05T14:41:08Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

greptile-apps · 2026-03-05T14:43:04Z

Greptile Summary

This PR restructures the audio dependency extras by introducing a new audio_common base group (shared between CPU and GPU paths) containing nemo_toolkit[asr], soundfile, torchaudio, onnx, and silero-vad, while audio_cpu and audio_cuda12 inherit from it and add their respective ONNX runtime backends. Cython and packaging are also added to the [dependency-groups] build group for build-time use.

Key findings from this review:

seaborn missing from all audio extras — the PR description explicitly lists seaborn as an ADV pipeline dependency, but it was not added to audio_common, audio_cpu, or audio_cuda12. Either the package is missing or the description needs to be corrected.
Several issues were raised in prior review rounds (unconstrained silero-vad, torchaudio, nvidia-cudnn-cu12, and torchcodec versions; torchcodec missing from [tool.uv.sources]; missing [tool.uv.conflicts] guard for audio_cpu/audio_cuda12; dropped platform marker on nvidia-cublas-cu12 in the lock file; transformers override potentially breaking nemo-toolkit[asr]; Cython placement discussion) — these remain open for resolution.

Confidence Score: 3/5

Not safe to merge — multiple unresolved P1 issues from prior rounds plus a newly found missing dependency (seaborn) that contradicts the PR description.

The prior review rounds identified several genuine P1 defects (dropped platform guard on nvidia-cublas-cu12, missing [tool.uv.sources] for torchcodec causing CPU-only wheel in a GPU extra, missing [tool.uv.conflicts] guard, unconstrained version specs for silero-vad/torchaudio/nvidia-cudnn-cu12/torchcodec, and the transformers override potentially breaking nemo-toolkit[asr]). This review adds another P1: seaborn is described as a required ADV pipeline dependency but is absent from all audio extras. The combination of unresolved prior P1s and this new gap keeps the score at 3.

pyproject.toml — multiple unconstrained version specs and missing seaborn; uv.lock — dropped platform marker on nvidia-cublas-cu12 and CPU-only torchcodec wheel resolved for the GPU extra.

Important Files Changed

Filename	Overview
pyproject.toml	Adds audio_common/audio_cpu/audio_cuda12 extras with new ADV pipeline deps; seaborn listed in PR description is absent from all audio extras
uv.lock	Lock file updated to include new transitive deps (coloredlogs, flatbuffers, humanfriendly, cython, silero-vad, onnx, onnxruntime-gpu, torchcodec 0.11.0 from PyPI CPU wheels)

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["nemo-curator[audio_cpu]"] --> B["nemo_curator[audio_common]"]
    C["nemo-curator[audio_cuda12]"] --> B
    C --> D["nemo_curator[cuda12]"]
    B --> E["nemo_toolkit[asr]>=2.7.2"]
    B --> F["soundfile>=0.12.0"]
    B --> G["torchaudio (unpinned)"]
    B --> H["onnx>=1.19.0"]
    B --> I["silero-vad (unpinned)"]
    A --> J["onnxruntime>=1.20.1,<1.24"]
    C --> K["onnxruntime-gpu>=1.20.1,<1.24"]
    C --> L["nvidia-cudnn-cu12 (unpinned)"]
    C --> M["torchcodec (unpinned, CPU wheel from PyPI)"]
    N["seaborn (in PR description but MISSING)"] -.->|"Should be in"| B
    style N fill:#ffcccc,stroke:#cc0000

_{Reviews (17): Last reviewed commit: "Merge branch 'main' into pr/audio-depend..." | Re-trigger Greptile}

greptile-apps · 2026-03-05T14:43:11Z

-    "nemo_curator[audio_cpu]",
+    "nemo_curator[audio_common]",
    "nemo_curator[cuda12]",
+    "nvidia-cudnn-cu12",


nvidia-cudnn-cu12 missing version constraint

onnxruntime-gpu has strict cuDNN version requirements that vary by release. For example, onnxruntime-gpu 1.20.x requires cuDNN 8.x or 9.x at a specific minor version. Leaving nvidia-cudnn-cu12 unconstrained means a newer incompatible cuDNN release could be installed, causing silent runtime failures or missing symbol errors at import time.

Please add a version pin or at minimum an upper bound (e.g. nvidia-cudnn-cu12>=9.1,<10), matching what onnxruntime-gpu>=1.20.1 expects.

Suggested change

"nvidia-cudnn-cu12",

"nvidia-cudnn-cu12>=9.1,<10",

greptile-apps · 2026-03-05T14:43:12Z

+audio_common = [
    "nemo_toolkit[asr]==2.4.0",
+    # ADV (Audio DataVerse) Dependencies
+    "Cython",


Cython is a build-time dependency, not a runtime dependency

Cython is a compiler/transpiler used at build time to generate C extensions — it is not a package that is imported at runtime. Listing it under [project.optional-dependencies] (runtime extras) is unconventional and may install an unnecessary compiler toolchain in production environments.

If it is genuinely needed at runtime (e.g. for a package that compiles extensions on first use), please add a comment explaining why. Otherwise, consider moving it to the [dependency-groups] build group or removing it from this group.

This seems valid but I don't know enough about build vs import time, @shubhamNvidia do you know which package ends up needing this a build time thing?

Cython is a build-time dependency needed by some transitive deps of nemo_toolkit[asr] (like texterrors). It's not imported at runtime by our code. I can verify if it's still needed or if it can be removed.

greptile-apps · 2026-03-05T14:43:13Z

+    "seaborn>=0.11.2",
+    "onnx>=1.19.0",
+    "torchcodec",
+    "silero-vad",


silero-vad missing version constraint

silero-vad has undergone significant API changes across its release history (e.g., changes to the loading API and model interface between 0.3.x, 0.4.x, and 5.x). Without a version pin or at least a lower bound, a future installation could pull in an incompatible version and break the ADV pipeline silently at runtime.

Consider pinning or bounding the version to match the API your code relies on, e.g.:

Suggested change

"silero-vad",

"silero-vad>=4.0,<6",

sarahyurick · 2026-03-05T20:04:28Z

Fyi @shubhamNvidia whenever the pyproject.toml file is changed, the uv.lock file needs to be updated too. You can do this by running uv lock within your Curator directory.

sarahyurick

uv.lock file needs to be updated by running uv lock within your Curator directory.

cc @thomasdhc who may be able to help with any dependency issues.

greptile-apps · 2026-03-25T13:45:12Z

+    "torchaudio",
+    "seaborn>=0.11.2",
+    "onnx>=1.19.0",
+    "torchcodec",


torchcodec missing from [tool.uv.sources]

All other PyTorch-ecosystem packages added as optional dependencies (torch, torchaudio, torchvision) have explicit source routing in [tool.uv.sources] to pull GPU-aware wheels from https://download.pytorch.org/whl/cu128 on x86_64 Linux:

torchaudio = [ { index = "pytorch", marker = "platform_machine == 'x86_64' and sys_platform != 'darwin'" }, { index = "pypi", marker = "platform_machine != 'x86_64' or sys_platform == 'darwin'" }, ]

torchcodec is a PyTorch-maintained package that similarly ships CUDA-specific wheels from the same PyTorch index. Without a matching entry in [tool.uv.sources], uv will fall back to PyPI (or the NVIDIA index) for torchcodec, which may resolve a CPU-only wheel, an incompatible CUDA build, or a version whose CUDA extension was not compiled against cu128. This would cause silent runtime failures for GPU-accelerated codec operations.

Please add a torchcodec entry to [tool.uv.sources] analogous to torchaudio/torchvision.

greptile-apps · 2026-03-25T13:45:14Z

+    # ADV (Audio DataVerse) Dependencies
+    "Cython",
+    "packaging",
+    "torchaudio",


torchaudio version constraint missing

torchaudio releases are strictly coupled to specific torch versions (e.g., torchaudio==2.5.0 requires exactly torch==2.5.0). Installing an unpinned torchaudio alongside torch (which is already a base dependency with no upper-bound) can result in a version mismatch that raises an ImportError or produces incorrect audio processing behaviour at runtime.

Consider adding a version constraint that mirrors what nemo_toolkit[asr]==2.4.0 (already in audio_common) expects, or at minimum align it with the torch version that NeMo 2.4.0 requires.

Suggested change

"torchaudio",

"torchaudio>=2.5.0,<2.6.0",

(replace the version range with whatever NeMo 2.4.0 expects)

greptile-apps · 2026-03-25T13:45:15Z

+    "torchaudio",
+    "seaborn>=0.11.2",
+    "onnx>=1.19.0",
+    "torchcodec",


torchcodec missing version constraint

torchcodec has no version constraint, meaning any release could be installed. Its C++ extension API changes between minor versions and its CUDA kernels are compiled against a specific torch version. Without a lower bound (and ideally an upper bound), a future installation could pull an incompatible release.

Suggested change

"torchcodec",

"torchcodec>=0.2.0,<0.4",

(adjust the range to match what the ADV pipeline has been validated against)

shubhamNvidia · 2026-03-25T13:53:42Z

@sarahyurick I’ve merged with main and resolved the conflicts—you can go ahead and run the Ruff linter now

sarahyurick · 2026-03-25T16:24:17Z

/ok to test c9e6752

Signed-off-by: shbhawsar <shbhawsar@nvidia.com>

shubhamNvidia · 2026-04-02T15:31:51Z

/ok to test 5d6f6d8

sarahyurick · 2026-04-02T18:12:41Z

/ok to test bf2ce78

greptile-apps · 2026-04-02T18:15:35Z

    "apex; sys_platform == 'never'",
    "distance; sys_platform == 'never'",
    "huggingface-hub>=0.34,<1.0", # Override huggingface-hub, transformers and data-designer require two different versions of hugging-face hub
+    "transformers>=4.56.0,<5.0", # Override nemo-toolkit[asr]'s <=4.52.0 constraint; required for Nemotron VL support


transformers override bypasses nemo-toolkit[asr]'s compatibility bound

nemo_toolkit[asr]>=2.7.2 (resolved to 2.7.2) declares transformers<=4.52.0 as a hard upper bound. This PR adds transformers>=4.56.0,<5.0 to override-dependencies, forcing the global resolution to 4.57.6 (visible in the lock file) — a version outside the range nemo-toolkit supports.

If any ASR code path inside nemo-toolkit 2.7.2 uses a transformers API that was removed, renamed, or semantically changed between 4.52.0 and 4.56.0, the audio pipeline will fail at runtime without any installation-time error. The PR comment attributes this override to "Nemotron VL support", which is unrelated to the audio extras, yet it silently invalidates the ASR dependency contract for all audio users.

Please verify that nemo-toolkit 2.7.2 ASR functionality is tested and confirmed working with transformers 4.56+, or pin the nemo_toolkit[asr] requirement to a release whose supported transformers range already covers 4.56+.

sarahyurick

LGTM. Awaiting a final review from @ayushdg as well.

ayushdg · 2026-04-02T18:57:07Z

/claude review

ayushdg

Couple of questions

ayushdg · 2026-04-02T18:52:25Z

    "apex; sys_platform == 'never'",
    "distance; sys_platform == 'never'",
    "huggingface-hub>=0.34,<1.0", # Override huggingface-hub, transformers and data-designer require two different versions of hugging-face hub
+    "transformers>=4.56.0,<5.0", # Override nemo-toolkit[asr]'s <=4.52.0 constraint; required for Nemotron VL support


This is no longer needed as we have bumped nemo toolkit. L267 has this logic in constraints already

You're right, the transformers override is redundant now that nemo-toolkit has been bumped. I'll remove it.

ayushdg · 2026-04-02T18:53:35Z

+audio_common = [
    "nemo_toolkit[asr]==2.4.0",
+    # ADV (Audio DataVerse) Dependencies
+    "Cython",


This seems valid but I don't know enough about build vs import time, @shubhamNvidia do you know which package ends up needing this a build time thing?

greptile-apps · 2026-04-02T19:20:52Z

 version = "9.10.2.21"
 source = { registry = "https://pypi.nvidia.com/" }
 dependencies = [
-    { name = "nvidia-cublas-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
+    { name = "nvidia-cublas-cu12" },
 ]
 wheels = [
+    { url = "https://pypi.nvidia.com/nvidia-cudnn-cu12/nvidia_cudnn_cu12-9.10.2.21-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:c9132cc3f8958447b4910a1720036d9eff5928cc3179b0a51fb6d167c6cc87d8" },
    { url = "https://pypi.nvidia.com/nvidia-cudnn-cu12/nvidia_cudnn_cu12-9.10.2.21-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:949452be657fa16687d0930933f032835951ef0892b37d2d53824d1a84dc97a8" },
+    { url = "https://pypi.nvidia.com/nvidia-cudnn-cu12/nvidia_cudnn_cu12-9.10.2.21-py3-none-win_amd64.whl", hash = "sha256:c6288de7d63e6cf62988f0923f96dc339cea362decb1bf5b3141883392a7d65e" },
 ]

 [[package]]


nvidia-cublas-cu12 platform guard silently dropped

Adding nvidia-cudnn-cu12 as an explicit requirement in audio_cuda12 caused uv to regenerate the nvidia-cudnn-cu12 lock entry with updated upstream metadata. As a result, the marker that previously restricted nvidia-cublas-cu12 to platform_machine == 'x86_64' and sys_platform == 'linux' was removed:

- { name = "nvidia-cublas-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" }, + { name = "nvidia-cublas-cu12" },

nvidia-cublas-cu12 ships only manylinux wheels (no Windows or macOS). Now that nvidia-cudnn-cu12 also has win_amd64 and aarch64 wheels in the lock, resolving audio_cuda12 on Windows will pull in nvidia-cudnn-cu12 and attempt to satisfy its nvidia-cublas-cu12 dependency — but no matching Windows wheel exists, producing a hard installation failure on that platform.

If Windows is an unsupported target for audio_cuda12, add a platform marker to the extra itself (e.g. ; sys_platform == 'linux'), or add nvidia-cublas-cu12 with an explicit Linux-only marker so the resolver doesn't attempt to install it on Windows.

shubhamNvidia · 2026-04-02T19:22:36Z

/ok to test 332e2c7

Signed-off-by: shbhawsar <shbhawsar@nvidia.com>

sarahyurick · 2026-04-02T20:56:44Z

/ok to test ebb4187

sarahyurick · 2026-04-02T23:19:40Z

/ok to test 5ec3af4

ayushdg · 2026-04-03T14:39:41Z

+    "Cython",
+    "packaging",
+    "torchaudio",
+    "seaborn>=0.11.2",


is this needed?

ayushdg · 2026-04-03T14:40:04Z

    "soundfile>=0.12.0",
+    # ADV (Audio DataVerse) Dependencies
+    "Cython",
+    "packaging",


Same concern about this

Cython and packaging: These are build-time prerequisites for nemo_toolkit[asr] — the [NeMo Sortformer model card] explicitly requires pip install Cython packaging before installing NeMo

They're not imported at runtime. I'll move them to [dependency-groups] build as suggested by @thomasdhc
seaborn: Not used anywhere, will remove.

thomasdhc · 2026-04-03T14:42:57Z

@shubhamNvidia Please move build time dependencies here. cc: @ayushdg

thomasdhc · 2026-04-03T14:44:58Z

 ]

-audio_cpu = [
+audio_common = [


Why make a distinct group for audio common? Audio cuda inherits from audio cpu

audio_common exists because audio_cpu and audio_cuda12 need different onnxruntime packages (onnxruntime vs onnxruntime-gpu). If audio_cuda12 inherited from audio_cpu, both CPU and GPU versions of onnxruntime would be installed, which can cause conflicts. audio_common shares everything except the onnxruntime variant.

thomasdhc · 2026-04-03T14:47:21Z

@shubhamNvidia The PR introduces a list of new dependencies, but there are no code changes that import these in this PR. Will there be a follow-up? Trying to understand the scope and requirements in this PR.

sarahyurick · 2026-04-03T15:02:44Z

Hi @thomasdhc this PR includes dependencies for the following implementations:

ayushdg · 2026-04-03T16:29:36Z

/ok to test 28cd599

Signed-off-by: shbhawsar <shbhawsar@nvidia.com>

sarahyurick · 2026-04-03T17:26:26Z

/ok to test 23a359b

thomasdhc

LGTM

github-actions Bot added the community-request label Mar 5, 2026

greptile-apps Bot reviewed Mar 5, 2026

View reviewed changes

sarahyurick self-requested a review March 5, 2026 18:10

ayushdg removed the community-request label Mar 5, 2026

sarahyurick requested changes Mar 9, 2026

View reviewed changes

sarahyurick requested a review from mohammadaaftabv March 24, 2026 17:23

shubhamNvidia requested a review from a team as a code owner March 25, 2026 13:42

greptile-apps Bot reviewed Mar 25, 2026

View reviewed changes

github-actions Bot added the community-request label Mar 25, 2026

copy-pr-bot Bot temporarily deployed to test March 25, 2026 16:24 Inactive

copy-pr-bot Bot had a problem deploying to nemo-ci March 25, 2026 16:24 Failure

shubhamNvidia added 3 commits April 2, 2026 15:07

Merge remote-tracking branch 'upstream/main' into pr/audio-dependencies

1758dcc

regenerate uv.lock

c313422

Signed-off-by: shbhawsar <shbhawsar@nvidia.com>

Merge upstream/main and resolve conflicts

47c1dbd

Signed-off-by: shbhawsar <shbhawsar@nvidia.com>

shubhamNvidia and others added 2 commits April 2, 2026 15:42

Merge remote-tracking branch 'upstream/main' into pr/audio-dependencies

5d6f6d8

Merge branch 'main' into pr/audio-dependencies

bf2ce78

greptile-apps Bot reviewed Apr 2, 2026

View reviewed changes

sarahyurick approved these changes Apr 2, 2026

View reviewed changes

This was referenced Apr 2, 2026

Add VADSegmentationStage for voice activity detection #1578

Merged

Add UTMOSFilterStage for audio quality filtering #1639

Merged

ayushdg reviewed Apr 2, 2026

View reviewed changes

greptile-apps Bot reviewed Apr 2, 2026

View reviewed changes

shubhamNvidia added 2 commits April 2, 2026 19:27

fix: remove redundant transformers override

332e2c7

Signed-off-by: shbhawsar <shbhawsar@nvidia.com>

Merge branch 'main' into pr/audio-dependencies

ebb4187

Merge branch 'main' into pr/audio-dependencies

5ec3af4

ayushdg reviewed Apr 3, 2026

View reviewed changes

thomasdhc reviewed Apr 3, 2026

View reviewed changes

shubhamNvidia and others added 2 commits April 3, 2026 16:39

fix: remove unused seaborn, move Cython and packaging to build deps

28cd599

Signed-off-by: shbhawsar <shbhawsar@nvidia.com>

Merge branch 'main' into pr/audio-dependencies

23a359b

thomasdhc approved these changes Apr 3, 2026

View reviewed changes

Uh oh!

Conversation

shubhamNvidia commented Mar 5, 2026

Uh oh!

copy-pr-bot Bot commented Mar 5, 2026

Uh oh!

greptile-apps Bot commented Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 3/5

Important Files Changed

Flowchart

Uh oh!

greptile-apps Bot Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

sarahyurick commented Mar 5, 2026

Uh oh!

sarahyurick left a comment

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

shubhamNvidia commented Mar 25, 2026

Uh oh!

sarahyurick commented Mar 25, 2026

Uh oh!

shubhamNvidia commented Apr 2, 2026

Uh oh!

sarahyurick commented Apr 2, 2026

Uh oh!

greptile-apps Bot Apr 2, 2026

Choose a reason for hiding this comment

Uh oh!

sarahyurick left a comment

Choose a reason for hiding this comment

Uh oh!

ayushdg commented Apr 2, 2026

Uh oh!

ayushdg left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Apr 2, 2026

Choose a reason for hiding this comment

Uh oh!

shubhamNvidia commented Apr 2, 2026

Uh oh!

sarahyurick commented Apr 2, 2026

Uh oh!

sarahyurick commented Apr 2, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot commented Mar 5, 2026 •

edited

Loading