Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

inflate: Read more bits when decoding #232

Merged
merged 5 commits into from
Feb 20, 2020
Merged

Conversation

klauspost
Copy link
Owner

@klauspost klauspost commented Feb 19, 2020

When a block is not marked as the final block it should be possible to read 10 further bits ahead without breaking the promise of not overreading.

Fixes #231

Also write blocks with only EOF as TYPE 01 (predefined tables) with only an EOB literal. This saves 3-4 bytes at no cost.

The smallest block seems to be a predefined block with a single EOB, which would be 10 bits + EOB from current block (current limit).

This should make it possible to fill more bits at the time when decoding.

λ benchcmp old.txt new.txt
benchmark                       old ns/op     new ns/op     delta
BenchmarkGunzipCopy-12          27830317      26617762      -4.36%
BenchmarkGunzipNoWriteTo-12     27878505      26705660      -4.21%

benchmark                       old MB/s     new MB/s     speedup
BenchmarkGunzipCopy-12          171.50       179.31       1.05x
BenchmarkGunzipNoWriteTo-12     171.20       178.72       1.04x

Fixes #231

When a block is not marked as the final block it should be possible to read 10 further bits ahead.

The smallest block seems to be a predefined block with a single EOB, which would be 10 bits + EOB from current block (current limit).

This should make it possible to fill more bits at the time when decoding.
@klauspost
Copy link
Owner Author

Maybe @dsnet can spot some obvious flaw. After all he had the great idea to extend it to the size of EOB.

If not, this would be a reasonable thing to upstream.

@klauspost
Copy link
Owner Author

Tried a few variations on reading. Didn't manage to find anything faster.

@klauspost klauspost merged commit 60e4844 into master Feb 20, 2020
@klauspost klauspost deleted the inflate-read-more-bits branch February 20, 2020 18:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

inflate: Read more bits if not final block
1 participant