Skip to content

Conversation

@eschnett
Copy link

No description provided.

@eschnett eschnett requested a review from a team as a code owner November 12, 2023 17:16
@braingram
Copy link
Contributor

Thanks for working on this. Adding blosc compression to asdf seems like a great addition.

There is support for adding compression algorithms via the Compressor extension api:
https://asdf.readthedocs.io/en/latest/asdf/extending/compressors.html
As the extension api appears to be sufficient for adding blosc support I'd rather add blosc via an extension than add it to the asdf library directly (to keep the core library lightweight but also to make sure the extension api is complete).

I recently started a package asdf-compression: https://github.com/braingram/asdf-compression
which adds blosc support. The blosc compressor in that package could use some of the improvements that you've included in this PR:
https://github.com/braingram/asdf-compression/blob/main/asdf_compression/blsc/compressor.py
asdf-compression will be moved to asdf-format soon and if it's easier I can make the move now and we can work on that package.

@eschnett
Copy link
Author

@braingram For Blosc decompression: Is there a way to convert b"".join(data) to a pointer, i.e. an integer that I can pass to the Blosc library? This would avoid copying data while decompressing.

@eschnett
Copy link
Author

Closing this in favour of braingram/asdf-compression#1

@eschnett eschnett closed this Nov 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants