Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

tune specific params in the base model #7745

Merged
merged 10 commits into from
Jan 16, 2024
Merged

tune specific params in the base model #7745

merged 10 commits into from
Jan 16, 2024

Conversation

arendu
Copy link
Collaborator

@arendu arendu commented Oct 18, 2023 •
edited
Loading

What does this PR do ?

tune only a selected subset of the params of the base model.

Collection: [NLP]

Changelog

  • Add specific line by line info of high level changes in this PR.

Usage

  • You can potentially add a usage example below
# Add a code snippet demonstrating how to use this

Before your PR is "Ready for review"

Pre checks:

  • Make sure you read and followed Contributor guidelines
  • Did you write any new necessary tests?
  • Did you add or update any necessary documentation?
  • Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
    • Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

  • New Feature
  • Bugfix
  • Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

Who can review?

Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.

Additional Information

  • Related to # (issue)

@github-actions github-actions bot added the NLP label Oct 18, 2023
@arendu arendu requested a review from Davood-M October 18, 2023 04:02
bmwshop
bmwshop reviewed Oct 18, 2023
View reviewed changes
Copy link
Collaborator

@bmwshop bmwshop left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@ertkonuk
Copy link
Collaborator

I think it looks good.

@arendu arendu requested a review from cuichenx October 27, 2023 20:15
@arendu
Copy link
Collaborator Author

arendu commented Nov 8, 2023

jenkins

@github-actions GitHub Actions
Copy link
Contributor

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

@github-actions github-actions bot added the stale label Nov 23, 2023
@github-actions GitHub Actions
Copy link
Contributor

This PR was closed because it has been inactive for 7 days since being marked as stale.

@github-actions github-actions bot closed this Nov 30, 2023
@arendu arendu reopened this Jan 10, 2024
@arendu
Copy link
Collaborator Author

arendu commented Jan 10, 2024

jenkins

cuichenx
cuichenx requested changes Jan 10, 2024
View reviewed changes
Copy link
Collaborator

@cuichenx cuichenx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you rename tunable_param_* to tunable_base_model_param_* and/or add a comment to specify it's for selective tuning? just because adapter weights are also tunable so it might be confusing

@arendu
renamed
482dde4 
Signed-off-by: arendu <[email protected]>
@arendu
Copy link
Collaborator Author

arendu commented Jan 10, 2024

jenkins

cuichenx
cuichenx previously approved these changes Jan 10, 2024
View reviewed changes
Copy link
Collaborator

@cuichenx cuichenx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, thanks!

@cuichenx
Copy link
Collaborator

jenkins

1 similar comment
@arendu
Copy link
Collaborator Author

arendu commented Jan 10, 2024

jenkins

@github-actions github-actions bot removed the stale label Jan 11, 2024
@arendu arendu requested a review from cuichenx January 13, 2024 19:34
@cuichenx
Copy link
Collaborator

jenkins

cuichenx
cuichenx approved these changes Jan 16, 2024
View reviewed changes
Copy link
Collaborator

@cuichenx cuichenx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@arendu arendu merged commit b404b84 into main Jan 16, 2024
15 checks passed
@arendu arendu deleted the adithyare/submodule branch January 16, 2024 17:07
jubick1337 pushed a commit that referenced this pull request Jan 17, 2024
@arendu @pre-commit-ci @jubick1337
tune specific params in the base model (#7745)
166adee 
* tune specific params in the base model

Signed-off-by: arendu <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* renamed

Signed-off-by: arendu <[email protected]>

* check for attr before calling

Signed-off-by: arendu <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: arendu <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
minitu pushed a commit to minitu/NeMo that referenced this pull request Jan 19, 2024
@arendu @pre-commit-ci
tune specific params in the base model (NVIDIA#7745)
4e9e05c 
* tune specific params in the base model

Signed-off-by: arendu <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* renamed

Signed-off-by: arendu <[email protected]>

* check for attr before calling

Signed-off-by: arendu <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: arendu <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
stevehuang52 pushed a commit that referenced this pull request Jan 31, 2024
@arendu @pre-commit-ci @stevehuang52
tune specific params in the base model (#7745)
75cf584 
* tune specific params in the base model

Signed-off-by: arendu <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* renamed

Signed-off-by: arendu <[email protected]>

* check for attr before calling

Signed-off-by: arendu <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: arendu <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: stevehuang52 <[email protected]>
ssh-meister pushed a commit to ssh-meister/NeMo that referenced this pull request Feb 15, 2024
@arendu @pre-commit-ci @sashameister
tune specific params in the base model (NVIDIA#7745)
77159c5 
* tune specific params in the base model

Signed-off-by: arendu <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* renamed

Signed-off-by: arendu <[email protected]>

* check for attr before calling

Signed-off-by: arendu <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: arendu <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Sasha Meister <[email protected]>
pablo-garay pushed a commit that referenced this pull request Mar 19, 2024
@arendu @pre-commit-ci @pablo-garay
tune specific params in the base model (#7745)
012bd83 
* tune specific params in the base model

Signed-off-by: arendu <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* renamed

Signed-off-by: arendu <[email protected]>

* check for attr before calling

Signed-off-by: arendu <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: arendu <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Pablo Garay <[email protected]>
rohitrango pushed a commit to rohitrango/NeMo that referenced this pull request Jun 25, 2024
@arendu @pre-commit-ci
tune specific params in the base model (NVIDIA#7745)
39d18f8 
* tune specific params in the base model

Signed-off-by: arendu <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* renamed

Signed-off-by: arendu <[email protected]>

* check for attr before calling

Signed-off-by: arendu <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: arendu <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cuichenx cuichenx cuichenx approved these changes

@meatybobby meatybobby Awaiting requested review from meatybobby

@ertkonuk ertkonuk Awaiting requested review from ertkonuk

@Davood-M Davood-M Awaiting requested review from Davood-M

@bmwshop bmwshop Awaiting requested review from bmwshop

Assignees
No one assigned
Labels
NLP
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@arendu @ertkonuk @cuichenx @bmwshop