chore(clip): update the clip example to make it compatible with HF format #472
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
awni
reviewed
Feb 21, 2024
awni
reviewed
Feb 21, 2024
awni
reviewed
Feb 21, 2024
awni
reviewed
Feb 21, 2024
awni
reviewed
Feb 21, 2024
Member
awni
left a comment
awni
left a comment
There was a problem hiding this comment.
Awesome!! Thank you! That is a big improvement.
I left a few minor comments, let me know what you think then we can merge.
Co-authored-by: Awni Hannun <[email protected]>
Merged
Contributor
Author
|
@awni, thanks for helping with reviewing the PR. I have addressed all the comments, it should be ready to merge now. Please let me know if there are any other changes needed. |
mzbac
commented
Feb 22, 2024
mzbac
commented
Feb 22, 2024
Member
|
@mzbac let me know when I should check this again |
Contributor
Author
|
@awni, this is ready for merging. Once this is merged, I will work with @nkasmanoff on #461 |
awni
reviewed
Feb 23, 2024
| if "position_ids" in k: | ||
| # Remove unused position_ids | ||
| continue | ||
| elif "patch_embedding.weight" in k: |
awni
reviewed
Feb 23, 2024
awni
reviewed
Feb 23, 2024
Member
|
Thanks!! |
devonthomas35
pushed a commit
to devonthomas35/mlx-examples
that referenced
this pull request
Mar 11, 2024
…rmat (ml-explore#472) * chore(clip): update the clip model to be HF format * Update clip/convert.py Co-authored-by: Awni Hannun <[email protected]> * chore: address comments * chore: rename ClipVisionModel and ClipTextModel * chore: add output hidden_states support * chore: remove custom conv2d and apply weight transpose during weight sanitizing * Update clip/model.py * Update clip/model.py --------- Co-authored-by: Awni Hannun <[email protected]>
junpeiz
pushed a commit
to junpeiz/mlx-examples
that referenced
this pull request
Jul 28, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Update the clip model format to HF format to make it easier for later use as a vision tower in the llava model.
close #463