Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Downgrading the 'datasets' package from 3.0.0 to 2.21.0 for Multilang_ASR.ipynb and ASR_CTC_Language_Finetuning.ipynb #11675

Merged
merged 2 commits into from
Dec 20, 2024

Conversation

weiqingw4ng
Copy link
Collaborator

@weiqingw4ng weiqingw4ng commented Dec 20, 2024

What does this PR do ?

'datasets' recently released the latest version 3.0.0 on Sep 11th, but some HF datasets (e.g., MCV 3.0, MCV 6.1) do not update their code for the latest version. Attempting to download these MCV datasets using datasets 3.0.0 results in the following error:
TypeError: DownloadConfig.__init__() got an unexpected keyword argument 'ignore_url_params'.

As a workaround, we need to downgrade the datasets package to version 2.21.0 in some tutorial notebooks. Please check the first block of each notebook for this change.

Collection: [Note which collection this PR will affect]
NeMo/tutorial/asr

Changelog

Adding !pip install datasets==2.21.0 in the first block of each tutorial.

GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

Before your PR is "Ready for review"

Pre checks:

  • Make sure you read and followed Contributor guidelines
  • Did you write any new necessary tests?
  • Did you add or update any necessary documentation?
  • Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
    • Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

  • New Feature
  • Bugfix
  • Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

Who can review?

Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.

Additional Information

Although unrelated to this PR, I wanted to highlight an issue encountered when calling Trainer.test() multiple times. In the Multilang_ASR.ipynb notebook, this sometimes results in a RuntimeError: CUDA error: an illegal memory access was encountered error in certain containers with specific PyTorch versions. This behavior has been verified in containers 24.06 and 24.12-rc0 (nvcr.io/nvidia/nemo:24.12-rc0), though it does not occur consistently and does not appear in other containers like 24.05 (nvcr.io/nvidia/nemo:24.05). A similar issue has been documented here with the 24.06 container (nvcr.io/nvidia/pytorch:24.06-py3). Testing is ongoing, but the root cause has not yet been identified.

@titu1994 titu1994 merged commit 27f451f into NVIDIA:main Dec 20, 2024
31 of 181 checks passed
ko3n1g pushed a commit that referenced this pull request Dec 20, 2024
…_ASR.ipynb and ASR_CTC_Language_Finetuning.ipynb (#11675)

* Downgrading the 'datasets' package from 3.0.0 to 2.21.0 for Multilang_ASR.ipynb

Signed-off-by: Weiqing Wang <[email protected]>

* Downgrading the 'datasets' package from 3.0.0 to 2.21.0 for ASR_CTC_Language_Finetuning.ipynb

Signed-off-by: Weiqing Wang <[email protected]>

---------

Signed-off-by: Weiqing Wang <[email protected]>
titu1994 pushed a commit that referenced this pull request Dec 20, 2024
…_ASR.ipynb and ASR_CTC_Language_Finetuning.ipynb (#11675) (#11677)

* Downgrading the 'datasets' package from 3.0.0 to 2.21.0 for Multilang_ASR.ipynb



* Downgrading the 'datasets' package from 3.0.0 to 2.21.0 for ASR_CTC_Language_Finetuning.ipynb



---------

Signed-off-by: Weiqing Wang <[email protected]>
Co-authored-by: Weiqing Wang <[email protected]>
BoxiangW pushed a commit that referenced this pull request Dec 20, 2024
…_ASR.ipynb and ASR_CTC_Language_Finetuning.ipynb (#11675)

* Downgrading the 'datasets' package from 3.0.0 to 2.21.0 for Multilang_ASR.ipynb

Signed-off-by: Weiqing Wang <[email protected]>

* Downgrading the 'datasets' package from 3.0.0 to 2.21.0 for ASR_CTC_Language_Finetuning.ipynb

Signed-off-by: Weiqing Wang <[email protected]>

---------

Signed-off-by: Weiqing Wang <[email protected]>
BoxiangW pushed a commit that referenced this pull request Dec 23, 2024
…_ASR.ipynb and ASR_CTC_Language_Finetuning.ipynb (#11675)

* Downgrading the 'datasets' package from 3.0.0 to 2.21.0 for Multilang_ASR.ipynb

Signed-off-by: Weiqing Wang <[email protected]>

* Downgrading the 'datasets' package from 3.0.0 to 2.21.0 for ASR_CTC_Language_Finetuning.ipynb

Signed-off-by: Weiqing Wang <[email protected]>

---------

Signed-off-by: Weiqing Wang <[email protected]>
abhinavg4 pushed a commit that referenced this pull request Jan 30, 2025
…_ASR.ipynb and ASR_CTC_Language_Finetuning.ipynb (#11675)

* Downgrading the 'datasets' package from 3.0.0 to 2.21.0 for Multilang_ASR.ipynb

Signed-off-by: Weiqing Wang <[email protected]>

* Downgrading the 'datasets' package from 3.0.0 to 2.21.0 for ASR_CTC_Language_Finetuning.ipynb

Signed-off-by: Weiqing Wang <[email protected]>

---------

Signed-off-by: Weiqing Wang <[email protected]>
Signed-off-by: Abhinav Garg <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants