-
Notifications
You must be signed in to change notification settings - Fork 195
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Explicit automatic alignment of header #178
Comments
No it's not overly specific, actually there's already a PR for that. I was waiting for more need for it before merging, but it seems this is picking up in low level frameworks where alignment could really help speed up load times. |
Also I love the project ! If you want pure rust ML framework I recommend https://github.com/coreylowman/dfdx (Still very early on, but there's at least a lot to inspire from IMO). For instance I implemented https://github.com/Narsil/fast_gpt2 (without dfdx, more like your approach, but still stealing the mkl bindings from dfdx to get the performance ! ) |
Thank you, @Narsil! Yeah, doing math low level isn't that popular apart from people who know how to code and are on "sub-par" HW by today's standards. Also thanks for the mention of |
It's still the future in my eyes. The ML fields is somewhat settling and not experimenting as much as it used to, performance is becoming a real concern for anything at scale. And all the python solution for performance are way too clunky to beat compiled code. |
Closing because #148 is merged |
My use case might be overly specific, but when writing/using vectorised code on mmap'd safetensors file header sometimes causes everything to have an odd-numbered pointer which breaks even 16 bit vectorisation. Is there a possibility that python bindings will get an option to save with an explicit alignment? Just padding the header should be enough for most use cases.
The text was updated successfully, but these errors were encountered: