Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change FP8 Defaults #7894

Merged
merged 1 commit into from
Nov 16, 2023
Merged

Change FP8 Defaults #7894

merged 1 commit into from
Nov 16, 2023

Conversation

cuichenx
Copy link
Collaborator

What does this PR do ?

Change the certain FP8 parameter defaults to more recommended values so that new users can find the optimal FP8 recipe faster.

Collection: NLP

Changelog

  • Change all defaults of fp8_amax_history_len from 1 to 1024
  • Change all defaults of fp8_amax_compute_algo from 'most_recent' to 'max'
  • Change config defaults of fp8_hybrid to True to mark that hybrid is the preferred fp8 mode (note fp8 is still False so by default FP8 is still off.)

Before your PR is "Ready for review"

Pre checks:

  • Make sure you read and followed Contributor guidelines
  • Did you write any new necessary tests?
  • Did you add or update any necessary documentation?
  • Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
    • Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

  • New Feature
  • Bugfix
  • Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

Who can review?

Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.

Additional Information

  • Related to # (issue)

Signed-off-by: Chen Cui <[email protected]>
Copy link
Collaborator

@ericharper ericharper left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks!

@ericharper
Copy link
Collaborator

jenkins

@cuichenx cuichenx merged commit b97ee35 into main Nov 16, 2023
15 checks passed
@cuichenx cuichenx deleted the chcui/better_fp8_defaults branch November 16, 2023 21:16
pzelasko pushed a commit to pzelasko/NeMo that referenced this pull request Jan 3, 2024
Signed-off-by: Chen Cui <[email protected]>
Signed-off-by: Piotr Żelasko <[email protected]>
rohitrango pushed a commit to rohitrango/NeMo that referenced this pull request Jun 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants