-
Notifications
You must be signed in to change notification settings - Fork 2.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[NeMo-UX] Add mistral-7b model #9066
Conversation
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
def state_transform( | ||
source_key: Union[str, Tuple[str, ...], Dict[str, str]], target_key: Union[str, Tuple[str, ...]], fn: F | ||
) -> StateDictTransform[F]: | ||
... |
Check notice
Code scanning / CodeQL
Statement has no effect Note
# with torch.device("meta"): | ||
# target = self.init() |
Check notice
Code scanning / CodeQL
Commented-out code Note
# for key in target.state_dict(): | ||
# if key.endswith("_extra_state"): | ||
# del target.state_dict()[key] |
Check notice
Code scanning / CodeQL
Commented-out code Note
def state_transform( | ||
source_key: Union[str, Tuple[str, ...], Dict[str, str]], target_key: Union[str, Tuple[str, ...]], | ||
) -> Callable[[F], StateDictTransform[F]]: | ||
... |
Check notice
Code scanning / CodeQL
Statement has no effect Note
Signed-off-by: Chen Cui <[email protected]>
Signed-off-by: Chen Cui <[email protected]>
for more information, see https://pre-commit.ci
Signed-off-by: Chen Cui <[email protected]>
Signed-off-by: Chen Cui <[email protected]>
Signed-off-by: Chen Cui <[email protected]>
for more information, see https://pre-commit.ci
Signed-off-by: Chen Cui <[email protected]>
waiting for #9065 to go in first |
for more information, see https://pre-commit.ci
Signed-off-by: Chen Cui <[email protected]>
* Adding MegatronParallel * Move over _strategy_liMegatronCheckpointIO * Adding GPTModel & MockDataModule * Adding mixed-precision to NeMo * Fix import * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding MegatronParallel * Move over _strategy_liMegatronCheckpointIO * Adding GPTModel & MockDataModule * Add nemo.io to MegatronStrategy * Move to cloudpickle * Adding Mistral7B model * Fix small bug inside state-transform * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * revert unintended changes Signed-off-by: Chen Cui <[email protected]> * clean up code and reinstate mix precision tests Signed-off-by: Chen Cui <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * clean up Signed-off-by: Chen Cui <[email protected]> * use cpu for unit test Signed-off-by: Chen Cui <[email protected]> * clean up Signed-off-by: Chen Cui <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix test Signed-off-by: Chen Cui <[email protected]> * mistral requires hf login so use a toy model for now Signed-off-by: Chen Cui <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * revert accidental change Signed-off-by: Chen Cui <[email protected]> --------- Signed-off-by: Chen Cui <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Chen Cui <[email protected]>
What does this PR do ?
Add a one line overview of what this PR aims to accomplish.
Collection: [Note which collection this PR will affect]
Changelog
Usage
# Add a code snippet demonstrating how to use this
Jenkins CI
The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.
There's no need to comment
jenkins
on the PR to trigger Jenkins CI.The GitHub Actions CI will run automatically when the PR is opened.
To run CI on an untrusted fork, a NeMo user with write access must click "Approve and run".
Before your PR is "Ready for review"
Pre checks:
PR Type:
If you haven't finished some of the above items you can still open "Draft" PR.
Who can review?
Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.
Additional Information