Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve GPU utilization in student training #783

Open
Tracked by #453
, #912 ,  #891
...
eu9ene opened this issue Jul 31, 2024 · 0 comments
Open
Tracked by #453
, #912 ,  #891
...

Improve GPU utilization in student training #783

eu9ene opened this issue Jul 31, 2024 · 0 comments
Labels
cost & perf Speeding up and lowering cost for the pipeline

Comments

@eu9ene
Copy link
Collaborator

eu9ene commented Jul 31, 2024

We noticed that it's only around 30%. It's likely because the model is smaller than the teacher. We can try improving it by increasing the batch size.

Screenshot 2024-07-31 at 1 33 51 PM

In comparison, for teacher training:

Screenshot 2024-07-31 at 1 37 12 PM
@eu9ene eu9ene added the cost & perf Speeding up and lowering cost for the pipeline label Jul 31, 2024
@eu9ene eu9ene mentioned this issue Jul 31, 2024
Open
@marco-c marco-c changed the title Improve GPU utilizaiton in student training Improve GPU utilization in student training Aug 7, 2024
@gregtatum gregtatum mentioned this issue Oct 30, 2024
Open
@eu9ene eu9ene mentioned this issue Feb 5, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
cost & perf Speeding up and lowering cost for the pipeline
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@eu9ene