Add 2bit packing routines #797
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/797
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit c781cbd with merge base e15e509 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
facebook-github-bot
added
the
CLA Signed
|
This pull request was exported from Phabricator. Differential Revision: D62133659 |
Summary: Pull Request resolved: pytorch#797 Adds 2-bit packing/unpacking routines, together with tests/benchmarks. This adds 24 new 2 bit kernels (3 tile sizes x 8 variants). Reviewed By: digantdesai Differential Revision: D62133659
|
This pull request was exported from Phabricator. Differential Revision: D62133659 |
metascroy
force-pushed
the
export-D62133659
branch
from
73618d8 to
c781cbd
Compare
manuelcandales
approved these changes
Sep 3, 2024
digantdesai
approved these changes
Sep 3, 2024
jerryzh168
pushed a commit
to jerryzh168/ao
that referenced
this pull request
Sep 4, 2024
Differential Revision: D62133659 Pull Request resolved: pytorch#797
yanbing-j
pushed a commit
to yanbing-j/ao
that referenced
this pull request
Dec 9, 2024
Update prompt chat is waiting for, which was modified by pytorch/torchchat#476 Modify logging defaults to not create a file in a temp folder without prompting user, but rather just print an info messages Replace few `prints` with `logging.info` This way, information about bandwith achieved will be printed to the console, but not to the web-browser chat window Test plan: ``` % python3 torchchat.py browser stories110M & % curl -L http://127.0.0.1:5000 % curl -d "prompt=Once upon a time" -X POST http://127.0.0.1:5000/chat ``` TODOs: - Add CI that repeats above steps -Figure out if spawning generator from the browser can be avoided Fixes pytorch/torchchat#785
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
Adds 2-bit packing/unpacking routines, together with tests/benchmarks.
This adds 24 new 2 bit kernels (3 tile sizes x 8 variants).
Reviewed By: digantdesai
Differential Revision: D62133659