-
Notifications
You must be signed in to change notification settings - Fork 694
Metal backend: store weights outside .so #15341
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/15341
Note: Links to docs will display an error until the docs builds have been completed. ❌ 3 New Failures, 3 Pending, 2 Unrelated FailuresAs of commit d8e6d13 with merge base 5d71c9b ( NEW FAILURES - The following jobs have failed:
BROKEN TRUNK - The following jobs failed but were present on the merge base:👉 Rebase onto the `viable/strict` branch to avoid these failures
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
user_input_placeholders.append(node.meta["val"]) | ||
|
||
# Base options for all devices | ||
options: dict[str, typing.Any] = { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a reason why we need metal_backend.py? It feels very much like cuda_backend.py
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, I agree. Next week we should begin refactoring, and moving most of this code to a new aoti_backend.py. Same with most of the code in metal_backend.cpp/cuda_backend.cpp
This pull request refactors how model weight constants are packaged and loaded in the Metal backend. The main change is separating weight constants from the shared object (
.so
) file, storing them as a binary blob on disk, and updating the runtime to load these weights from the blob. This speeds up loading time.