Skip to content

commit dislora #11122

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 2 commits into
base: develop
Choose a base branch
from
Open

commit dislora #11122

wants to merge 2 commits into from

Conversation

Pioneer-wxh
Copy link

@Pioneer-wxh Pioneer-wxh commented Sep 26, 2025 •
edited
Loading

Before submitting

  • Lint code. If there are lint issues, please format the code first.
# Install and register `pre-commit` in the project folder
pip install pre-commit && pre-commit install

# Process previous code files separately
pre-commit run --file XXXX.py
  • Add test cases into tests folder. If there are codecov issues, please add tests cases first.

PR types

PR changes

Description

@CLAassistant
Copy link

CLAassistant commented Sep 26, 2025 •
edited
Loading

CLA assistant check
All committers have signed the CLA.

lugimzzz
lugimzzz reviewed Oct 16, 2025
View reviewed changes
llm/run_finetune.py

train_ds, dev_ds, test_ds = create_dataset(data_args, training_args)

train_dataset_size = None
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

为什么需要get datasize?

paddlenlp/trl/sft_trainer.py Outdated
if max_eval_iters > 0 and step >= max_eval_iters - 1:
break

def _calc_ortho_loss(self, model):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

不要写在SFTTrainer里，peft/dislora目录下新起一个DisLoRATrainer

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@lugimzzz lugimzzz lugimzzz left review comments

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Pioneer-wxh @CLAassistant @lugimzzz