New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix FSDP SHARP integration #8170

Closed

janEbert wants to merge 1 commit into NVIDIA:main from janEbert:fsdp-sharp

Contributor

janEbert commented Jan 13, 2024

The attribute was never implemented.

What does this PR do ?

For NeMo NLP, the FSDP sharding strategy cannot be used currently because the sharp attribute for the NLPFSDPStrategy was not actually implemented in #7793, even though it is queried.

Collection:

NLP

Changelog

Implement missing NLPFSDPStrategy.sharp attribute.

Jenkins CI

To run Jenkins, a NeMo User with write access must comment jenkins on the PR.

Before your PR is "Ready for review"

Pre checks:

Make sure you read and followed Contributor guidelines
Did you write any new necessary tests?
Did you add or update any necessary documentation?
Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

New Feature
Bugfix
Documentation

Who can review?

Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.

Additional Information

Related to #7793 (pull request)


          Fix FSDP SHARP integration

437252f

The attribute was never implemented.

github-actions bot added the NLP label

Contributor

github-actions bot commented Jan 28, 2024

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

github-actions bot added the stale label

Contributor Author

janEbert commented Jan 28, 2024

Bla.

github-actions bot removed the stale label

Contributor

github-actions bot commented Feb 13, 2024

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

github-actions bot added the stale label

Contributor Author

janEbert commented Feb 13, 2024

Bla.

github-actions bot removed the stale label

Contributor

github-actions bot commented Feb 29, 2024

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

github-actions bot added the stale label

Contributor Author

janEbert commented Feb 29, 2024

Bla.

github-actions bot removed the stale label

Contributor

github-actions bot commented Mar 16, 2024

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

github-actions bot added the stale label

Contributor Author

janEbert commented Mar 18, 2024

Bla.

github-actions bot removed the stale label

Contributor

github-actions bot commented Apr 2, 2024

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

github-actions bot added the stale label

Contributor Author

janEbert commented Apr 2, 2024

Bla.

github-actions bot removed the stale label

Contributor

github-actions bot commented Apr 17, 2024

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

github-actions bot added the stale label

Contributor Author

janEbert commented Apr 22, 2024

Bla.

Contributor Author

janEbert commented May 29, 2024

Independently fixed in dd69c7a.

janEbert closed this

janEbert deleted the fsdp-sharp branch

May 29, 2024 14:17

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment