Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revised compress method #1

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Conversation

lafncow
Copy link

@lafncow lafncow commented Apr 5, 2013

I made 2 changes to the "compress" method:

  1. it will return fewer than the target number of bytes if it is given a digest that is smaller than the target size already (instead of throwing an error)
  2. it spreads the modulo bytes around rather than dumping them all into the final byte (I think this might preserve some entropy, no?)

Adam Cornille added 2 commits April 5, 2013 16:08
Instead of throwing an error or zero-padding, "compress" now returns the
input bytes if there are less than or equal to "target" number of them.
I think this is logical since the goal of compress is to reduce the
complexity of the digest before making it human consumable. In this case
the complexity is already low enough to proceed.
Excess bytes are now distributed amongst the compressed bytes, instead
of being dumped into the final bit as they were before.
@blag
Copy link

blag commented Apr 12, 2017

I'm maintaining a Python 3 fork of humanhash on GitHub and PyPI.

Can you add some comments to the code to explain what this is doing? And why it's better than the existing compress method? Sorry to dig this up from four years ago...

@lafncow
Copy link
Author

lafncow commented May 9, 2017

Happy to resurrect this! Compression method comments are added.

Why is this better?
The old method divided the bytes into the target number of segments and after even division, placed all remainder bytes into the final segment. This meant that the effect of the remainder bytes on overall entropy was confined to the final byte.
In the new method, the remainder bytes are selected throughout the input bytes and are distributed evenly among the target segments, allowing them to express more entropy. The compression per input byte is more even, since the biggest difference in the number of input bytes per output byte is 1.

For example:

compress_old([123,456,798,147], 4)
# -> [123, 456, 789, 147]
compress_old([123,456,789,147,258,369,321],4)
# -> [123, 456, 789, 417] (only the last byte has changed)

compress_new([123,456,798,147], 4)
# -> [123, 456, 789, 147]
compress_new([123,456,789,147,258,369,321],4)
# -> [435, 902, 115, 321] (all 4 bytes have changed)

As an aside, I have an equivalent compress method prepared for the Javascript port and I will create a pull request there if this is merged.

Thanks!

@blag
Copy link

blag commented May 9, 2017

I'm maintaining the humanhash3 PyPI package, and if you can create a PR to my repo I'd be happy to merge it in. Thanks for the explanation! 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants